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OPERATIONAL  TESTING:  ENSURING  BETTER 
WEAPONS  FOR  OUR  TROOPS 


TUESDAY,  MARCH  22,  1994 

U.S.  Senate, 
Subcommittee  on  Federal  Services, 

Post  Office,  and  Civil  Service, 
Committee  on  Governmental  Affairs, 

Washington,  DC. 
The  Subcommittee  met,  pursu£int  to  notice,  at  10:41  a.m.  in  room 
SD-342,  Dirksen  Senate  Office  Building,  Hon.  David  Pryor,  Chair- 
man of  the  Subcommittee,  presiding. 
Present:  Senators  Pryor  and  Roth, 

OPENING  STATEMENT  OF  SENATOR  PRYOR 

Senator  Pryor.  Ladies  and  gentlemen,  our  meeting  will  come  to 
order.  We  waint  to  welcome  each  of  our  witnesses  this  morning  and 
our  colleague.  Senator  Roth. 

I  have  a  statement  that  I  am  not  going  to  take  the  time  to  read 
in  its  entirety.  By  the  way,  I  do  apologize  for  my  lateness  this 
morning.  Another  meeting  was  called  and  I  had  to  attend  it, 

I  would  like  to  say  that  for  about  10  years  my  colleague.  Senator 
Roth  of  Delaware,  and  myself  have  been  involved  in  promoting  a 
concept  of  operational  testing,  testing  weapons  before  we  buy  them. 
We  call  this  fly  before  you  buy,  I  would  like  to  also  state  that  since 
we  have  been  involved  on  this  issue  for  the  past  10  years — I  can't 
speak  for  Senator  Roth,  but  I  will  speak  for  myself— I  have  yet  dur- 
ing this  decade  of  working  on  the  issue  of  operational  testing,  mak- 
ing sure  that  weapons  work  before  we  buy  them — I  have  yet  to 
have  one  lobbyist  come  into  my  office  or  to  meet  me  in  the  hall  to 
say  we  need  to  make  sure  that  our  weapons  work.  There  has  not 
been  one, 

I  have  had  lobbyists  on  the  acquisition  side  of  our  economy  rep- 
resenting the  Department  of  Defense  and  contractors,  who  con- 
stantly come  to  my  office  and  say,  we  want  a  certain  weapon  to  be 
built  in  a  certain  town  or  in  a  certain  State,  We  all  know  those 
forces;  Senator  Roth  knows  those  forces.  No  one  has  come  to  say 
let  us  make  sure  they  work  before  we  buy  them, 

I  think  that  the  forces  of  acquisition  and  procurement  are  very 
strong  today,  strong  enough  that  operational  testing,  even  in  this 
administration,  has  become  a  very  low  priority,  I  hate  to  say  that; 
I  truly  hate  to  see  it. 


(1) 


I  would  like  to  point  to  the  chart  i  we  have  showing.  During  the 
decade  since  this  office  has  been  created,  an  office  designed  to  pre- 
vent a  student  from  grading  their  own  exam  papers,  this  office  has 
been  without  a  director  30  percent  of  the  time.  That  is  abysmal. 
Even  in  this  administration,  this  office  has  yet  to  be  filled  with  a 
Director  of  Operational  Testing.  This,  to  me,  is  a  sad  admission,  a 
sad  statement,  but  it  is  true. 

In  1983,  when  Senator  Roth  and  I  began  this  process  we  felt 
truly  that  the  Pentagon  was  allowing  the  students  to  grade  their 
own  exams,  so  to  speak,  by  allowing  the  acquisition  community  to 
oversee  operational  testing.  This  was  a  deplorable  practice,  a  prac- 
tice that  contributed  to  the  waste  of  billions  and  bilUons  of  dollars 
on  weapons  that  did  not  work,  that  today  are  in  warehouses  or 
have  been  junked. 

Even  worse,  many,  many  faulty  weapons  were  being  sent  to  our 
troops  for  use  in  combat.  Senator  Roth  and  I  are  going  to  continue 
lobbying  for  those  troops  and  ensuring  that  the  weapons  that  go  to 
them  work  and  do  not  endanger  their  lives. 

The  primary  accomplishment  of  this  1983  testing  legislation  was 
the  removal  of  on-going  and  potential  conflicts  of  interest  in  weap- 
ons testing.  We  did  this  by  requiring  by  law  that  the  Department 
of  Defense  practice  and  promote  greater  independence  when  testing 
the  weapons  we  provide  for  our  troops. 

Unfortunately,  Senator  Roth  and  I  had  a  wish  for  a  strong,  inde- 
pendent testing  office.  It  was  not  granted  overnight;  it  was  not 
easy.  For  the  first  18  months,  this  office  went  without  a  director. 
Even  after  the  position  was  filled,  continuing  Pentagon  animosity 
toward  this  office  severely  inhibited  it's  abihty  to  get  involved  early 
and  often  in  the  acquisition  process.  Senator  Roth  and  I  envisioned 
an  independent  testing  office  that  would  work  together  with  top  ac- 
quisition decisionmakers  to  identify  problems  early  in  the  life  of  a 
weapons  system. 

Let  me,  if  I  might,  allude  to  this  chart  here.  The  ASPJ  Jammer, 
a  radar  jammer  which  is  today  being  installed  on  our  F-14  fight- 
ers, which  is  today  being  sold  to  our  alUes,  has  never  passed  oper- 
ational testing.  It  is  incredible.  Ninety-five  of  the  376  systems  have 
already  been  completed,  and  the  system  has  never  passed  oper- 
ational testing. 

On  the  C-17 — 40  out  of  the  120  aircraft,  33  percent  were  pro- 
duced before  being  operationally  tested.  What  happens?  We  have 
seen  massive  cost  overruns,  and  the  C-17  does  not  meet  aU  of  its 
requirements.  The  B-IB  bomber— 100  percent  of  these  bombers 
were  produced  without  any  operationally  testing.  Well,  we  know 
the  story.  The  first  B-IB  bomber,  I  think,  that  was  tested  hit  a  pel- 
ican and  crashed.  The  rest  of  them,  I  think,  are  in  cold  storage. 

In  the  past  10  years  we  have  frequently  heard  complaints  from 
the  acquisition  community  £ind  the  forces  that  say  buy,  buy,  buy, 
that  testing  is  more  trouble  than  it  is  worth,  is  too  expensive,  slows 
down  the  process,  and  is  inconsistent  with  DOD  acquisition  strate- 
gies. We  will  show  you  another  chart  later  in  the  hearing  that  will 
demonstrate  how  very  Uttle  of  the  overall  costs  of  the  weapons  sys- 
tem is  used  in  operational  testing. 


1  See  page  51. 


This  office  has  been  isolated,  and  the  lack  of  the  support  that  it 
has  been  given  in  the  past  is  disturbing.  Unfortunately,  the  past 
14  months,  I  must  admit  to  you,  have  been  less  then  encouraging. 
The  Pentagon's  slowness  in  naming  a  testing  czar  is  really  taking 
the  life  out  of  our  independent  testers  and  has  relegated  this  agen- 
cy to  almost  a  lame  duck  status.  What  is  most  disturbing  is  that 
without  a  forceful  champion  for  independent  operational  testing, 
very  important  policy  decisions  are  taking  place  every  day  in  the 
Pentagon  without  the  benefit  of  this  office's  input. 

We  are  at  a  crossi:oads,  I  think,  today  in  the  way  we  use  oper- 
ational testing.  The  challenge  that  we  face,  and  the  reason  for  this 
hearing  today  is  to  see  what  we  have  done  for  the  last  decade,  to 
see  what  we  are  doing  now,  and  to  see  what  we  have  to  do  to  en- 
sure that  these  weapons  that  we  purchase  and  that  we  send  to  the 
field  actually  work  and  do  not  endanger  the  lives  of  our  fighting 
men  and  women. 

I  am  going  to  put  my  entire  statement  in  the  record. 

PREPARED  STATEMENT  OF  SENATOR  PRYOR 

This  year  the  Congress  is  once  again  taking  on  the  difficult  task  of  reforming  the 
massive  acquisition  system  within  the  Department  of  Defense,  as  part  of  a  broader 
effort  to  streamUne  Federal  procurement.  As  a  member  of  the  Senate  Governmental 
Affairs  Committee,  I  have  been  closely  monitoring  this  important  issue.  Thanks  to 
the  leadership  of  the  Clinton  Administration  and  my  distinguished  colleagues  in  the 
Congress — Senators  Nunn,  Glenn,  Levin  and  others  who  have  sponsored  S.  1587,  we 
are  about  to  provide  many  needed  and  long  awaited  reforms  that  will  significantly 
change  the  way  our  government  does  business. 

While  I  support  this  important  initiative,  myself  and  others  have  expressed  strong 
reservations  about  enacting  the  provisions  of  S.  1587  that  deal  with  the  Department 
of  Defense's  vital  responsibility  of  testing  weapons.  As  a  result,  today's  hearing  has 
been  called  to  focus  our  attention  on  the  importance  of  upholding  independent  weap- 
ons testing  in  the  Department  of  Defense. 

Independent  operational  testing  is  the  crucial  step  in  the  high  stakes,  high  dollar 
world  of  military  procurement  that  our  troops  simply  cannot  live  without.  In  1983, 
Senator  Roth  and  I  became  fed  up  with  the  Defense  Department's  continuing  oppo- 
sition to  making  operational  tesing  fully  independent  of  the  DOD  acquisition  com- 
munity. Despite  the  Pentagon's  objections.  Senator  Roth  and  I  successfully  passed 
legislation  in  Congress  creating  the  Pentagon's  first  independent  operational  testing 
office,  known  as  the  DOT&E.  This  office  was  designed  to  be  located  in  the  Office 
of  the  Secretary  of  Defense  and  to  remain  independent  fi:om  the  intense  pressures 
that  were  often  applied  by  the  acquisition  community. 

In  1983,  Senator  Roth  and  I  found  that  the  DOD's  process  for  acquiring  weapons 
was  hopelessly  riddled  with  conflicts  of  interest.  We  felt  the  Pentagon  was  "allowing 
the  students  to  grade  their  own  exams"  by  allowing  the  acquisition  community  to 
oversee  operational  testing.  This  deplorable  practice  contributed  to  the  waste  of  bil- 
lions of  dollars  on  weapons  that  did  not  work.  Even  worse,  faulty  weapons  were 
being  sent  to  our  troops  for  use  in  combat. 

The  primary  accomplishment  of  our  1983  testing  legislation  was  the  removal  of 
on  going  and  potential  conflicts  of  interest  in  weapons  testing.  We  did  this  by  re- 
quiring by  law  that  the  Department  of  Defense  practice  and  promote  greater  inde- 
pendence when  testing  the  weapons  we  provide  for  our  troops. 

Unfortunately,  our  wish  for  strong,  independent  testing  was  not  granted  over- 
night. The  Pentagon  leadership  opposed  our  legislation  and  then  took  19  months  to 
appoint  a  testing  director.  Even  after  the  personnel  were  finally  hired,  continuing 
Pentagon  animosity  toward  operational  testing  severely  inhibited  the  DOT&E's  abil- 
ity to  get  involved  early  and  often  in  the  acquisition  process. 

Senator  Roth  and  I  envisioned  an  independent  testing  office  that  would  work  to- 
gether with  top  acquisition  decision  makers  to  identify  problems  early  in  the  life  of 
a  weapon  system.  We  hoped  that  such  practices  would  enable  us  to  keep  bad  weap- 
ons away  from  our  troops,  and  of  course,  prevent  us  firom  spending  bilUons  of  dollars 
on  bad  weapons. 


Unfortunately,  the  DOT&E  is  currently  isolated  on  the  acquisition  perimeter  as 
the  "final  exam"  for  weapons.  As  a  result,  the  DOT&E  is  often  forced  to  either  "stop 
the  show"  or  feebly  object  as  faulty  weapons  are  forced  into  production. 

I  can  assure  everyone  in  this  hearing  room  that  we  did  not  intend  for  the  DOT&E 
to  be  either  a  "final  exam"  or  a  "show  stopper".  However,  over  the  past  10  years, 
two  key  factors  have  contributed  to  the  isolation  of  the  DOT&E  and  the  limits  on 
this  office's  abiUty  to  positively  impact  the  way  the  Pentagon  builds  weapons. 

The  first  factor  is  the  intolerance  of  many  in  the  acquisition  community  for  the 
existence  and  involvement  of  independent  operational  testing.  It  is  widely  known 
that  the  developers  and  advocates  of  new  weapons  prefer  little  or  no  interference 
by  operational  testers,  especially  when  performance  problems  exist. 

This  adversarial  relationship  between  the  operational  testing  and  acquisition  com- 
munities will  always  exist  to  some  degree.  However,  the  past  10  years  have  taught 
us  that  when  operational  testing  is  neglected,  the  delicate  balance  of  this  adversar- 
ial relationship  becomes  dangerously  tilted  in  the  favor  of  the  acquisition  commu- 
nity— thus  jeopardizing  the  very  existence  of  independent  OT&E. 

In  the  past  10  years,  we  have  fi-equently  heard  complaints  firom  the  acquisition 
community  that  testing  is  more  trouble  than  it  is  worth,  is  too  expensive,  slows 
down  the  process,  is  often  inconsistent  with  DOD  acquisition  strategies.  However, 
it  has  been  proven  that  operational  testing,  when  fully  integrated  in  the  acquisition 
process,  saves  time,  money,  and  most  importantly,  the  lives  of  our  fighting  forces. 
It  has  also  been  argued  that  the  contractors  should  be  allowed  more  involvement 
in  operational  testing.  However,  independent  testing  is  built  on  a  solid  foundation 
that  is  designed  to  limit  the  role  of  contractors  who  are  financiallv  attached  to  the 
weapons  being  developed.  We  cannot  revert  back  to  the  days  when  the  students 
graded  their  own  exams. 

Unfortunately,  the  testing  provisions  of  S.  1587  represent  these  misguided  criti- 
cisms of  independent  testing.  I  am  strongly  opposed  to  these  provisions,  and  I  hope 
this  hearing  will  shed  light  on  the  need  to  strengthen,  not  weaken,  independent 
testing  in  the  Pentagon.  ^^r„„„  .      ■,      j    ..    i.- 

The  second  factor  contributing  to  the  isolation  of  the  DOT&E  is  the  disturbing 
lack  of  support  for  independent  testing  by  the  leadership  of  the  Pentagon — dating 
back  to  the  creation  of  the  DOT&E  in  1983.  In  the  absence  of  strong  Pentagon  sup- 
port, we  have  jeopardized  our  independent  testers'  ability  to  ensure  that  the  tax- 
payers get  what  they  pay  for  and  that  our  troops  get  weapons  they  can  rely  on  in 
combat.  Our  operational  testers  must  receive  strong  support  fi-om  the  highest  levels 
of  the  Pentagon. 

Although  past  administrations  failed  to  provide  independent  testers  with  the  nec- 
essary support,  I  must  admit  that  I  had  very  high  hopes  for  the  current  DOD  lead- 
ership. Unfortunately,  the  past  14  months  have  been  less  than  encouraging. 

The  Pentagon's  slowness  in  naming  a  testing  czar  has  taken  the  life  out  of  our 
independent  testers  and  relegated  the  DOT&E  office  to  a  "lame  duck"  status. 

I  am  alarmed  by  the  Pentagon's  inability  to  take  operational  weapons  testing  off 
the  back-burner.  What  is  most  disturbing  is  that  without  a  forceful  champion  for 
independent  operational  testing,  important  policy  decisions  are  taking  place  every 
day  in  the  Pentagon  without  the  benefit  on  the  DOT&E's  input  and  guidance.  For 
example,  had  a  test  director  been  at  the  table  during  critical  discussions  on  how  to 
resolve  the  ASPJ  radar  jammer  disaster,  I  am  certain  that  the  situation  would  have 
been  handled  differently,  and  we  would  not  be  installing  faulty  systems  on  our  F- 
14  aircraft  or  selling  them  to  our  allies. 

In  addition,  had  a  test  director  been  at  the  table  last  year,  the  operational  testing 
office  probably  would  not  have  received  a  demotion.  In  last  year's  Defense  Author- 
ization bill,  the  DOT&E  was  stripped  of  its  authority  to  report  directly  to  the  Sec- 
retary of  Defense,  thus  sending  a  dangerous  signal  that  independent  testing  had  de- 
creased in  importance. 

Finally,  and  perhaps  most  importantlv,  had  a  test  director  been  at  the  table  dur- 
ing discussions  on  acquisition  reform,  the  legislation  currently  circulating  probably 
would  not  have  surfaced.  This  legislation  is  designed  to  undermine  independent 
testing  and  should  not  be  enacted.  . 

These  are  all  examples  of  what  happens  when  the  Pentagon  operates  without  the 
benefit  of  a  test  director's  input  and  guidance.  I  am  afraid  that  we  might  soon  learn 
of  other  expensive  lessons  that  could  have  been  avoided  if  an  honest,  aggressive, 
and  independent  test  director  had  been  confirmed  and  at  the  table  during  the  past 
14  months. 

After  ten  years  of  hard  fought  survival,  the  concept  of  independent  operational 
testing  has  reached  the  crossrwids  this  vear.  The  direction  we  choose  for  this  impor- 
tant function  will  significantly  impact  the  way  weapons  are  built  in  the  future.  The 
choices  are  simple — we  can  either  weaken  operational  testing  or  we  can  work  to 


strengthen  its  ability  to  encourage  the  development  of  cost-efTective  weapons  that 
work. 

Senator  Pryor.  I  will  now  yield  to  my  good  friend  and  colleague, 
Senator  Roth. 

OPENING  STATEMENT  OF  SENATOR  ROTH 

Senator  Roth.  Well,  thank  you,  Mr.  Chairman.  Like  you,  I  am 
frankly  shocked  at  the  lack  of  progress  that  has  been  made  the  last 
10  years,  and  I  am  shocked  because  it  seems  to  me  the  principal 
concern  we  have  is  assuring  that  our  fighting  men  and  women 
have  weapons  that  do  what  they  are  supposed  to  do,  a  very  simple 
proposition — fly  before  you  buy. 

The  policy  states  that  a  weapon  should  not  be  produced  until 
testing  shows  that  it  works.  In  the  B-IB,  as  you  pointed  out,  100 
percent  doesn't  work.  Look  at  the  time  eind  the  effort  and  the 
money  that  would  have  been  saved  if  this  simple  proposition  of  fly 
before  you  buy  had  applied  to  that  acquisition. 

Mr.  Chairman,  in  a  system  where  bureaucratic  interests  carry 
more  weight  than  results,  realistic  tests,  in  my  judgment,  and  I 
know  in  your  judgment  as  well,  are  vital  to  make  sure  weapons 
work  before  they  are  given  to  our  troops  on  the  battlefield. 

I  have  to  tell  you  I  am  very  concerned  about  the  Pentagon's  at- 
tempt— and  fraiJdy  it  is  a  bipartisan  attempt  to  dodge  because  it 
was  true  before  this  administration  and  it  is  true  today— to  dodge 
the  testing  process  by  a  very  simple  method,  and  that  is  to  erase 
the  independence  of  the  Director  of  Operational  Testing. 

An  interview  published  in  yesterday's  issue  of  Defense  News 
shows  that  the  Pentagon  is  playing  a  game  of  hot  potato  with  the 
Office  of  Operational  Testing.  In  that  interview,  the  Defense  De- 
partment comptroller  proposed  that  the  director  report  to  someone 
in  the  acquisition  msinagement  chain.  In  addition,  the  Defense  De- 
partment 1994  Annual  Report  to  Congress  contained  an  organiza- 
tional chart  that  had  the  Director  of  Operational  Testing  reporting 
to  the  Under  Secretary  of  Defense  for  Acquisition. 

Mr.  Chairman,  that  violates  the  whole  concept  of  independence 
outside  of  the  acquisition  chain  so  that  there  is  an  independent 
analysis  and  evaluation  made  as  to  whether  the  weapon  is  going 
to  fiinction  as  intended.  All  I  can  say  is  that  the  proposed  organiza- 
tion, in  my  judgment,  is  illegal.  The  law  says  that  the  Director 
must  be  independent  of  the  Under  Secretary  of  Defense  for  Acquisi- 
tion. 

Like  you,  I  am  concerned  that  the  Office  of  Operational  Testing 
has  not  had  a  director  the  last  14  months.  Without  a  director,  there 
is  no  one  in  the  Pentagon  to  fight  against  those  that  want  to  pro- 
cure the  weapons  even  if  they  do  not  work,  and  let  me  tell  you  that 
is  a  tremendous  force  in  the  Pentagon. 

Almost  each  month  last  year,  the  GAO  identified  weapons  pro- 
grams that  are  not  in  compliance  with  the  testing  laws.  Just  2 
weeks  ago,  I  received  a  GAO  report  on  a  special  computer  system 
that  the  Army  is  buying,  despite  the  fact  that  it  failed  its  oper- 
ational testing.  As  the  chart  shows,i  current  Pentagon  practice  al- 
lows anywhere  from  25  percent,  which  you  referred  to  in  your  com- 


1  See  page  50. 


ments,  Mr.  Chairman,  to  100  percent  of  the  production  to  be  com- 
pleted before  the  Pentagon  knows  if  a  system  works.  Of  course,  in 
the  case  of  the  B-IB  it  didn't  work.  This  is  clearly  at  odds  with 
the  Congressional  poUcy  of  fly  before  you  buy. 

Now,  the  condition  of  live-fire  testing  is  also  disconcerting.  All 
major  aircraft  being  developed  by  the  Pentagon  is  avoiding  tests  for 
survivability  and  ^ilnerability.  Mr.  Chairman,  in  the  recent  hear- 
ings on  our  procurement  reform  bills,  the  GAO  and  Defense  IG  op- 
posed the  provisions  of  the  Glenn  bill  that  would  weaken  the  cur- 
rent law. 

Mr.  Chairman,  I  won't  read  my  fiill  statement.  I  would  ask  that 
it  be  included  as  read,  but  just  let  me  point  out  that  I  still  believe, 
as  do  you,  that  live-fire  and  operational  testings  are  keys  to  judg- 
ing results.  I  might  say  that  my  proposal  for  defense  acquisition  re- 
form would  integrate  such  testing  early  in  the  acquisition  system. 
I  think  this  could  cause  a  reduction  of  costs  associated  with  finding 
acquisition  problems  late. 

I  think  there  is  a  strong  push  for  Congress  to  enact  procurement 
reform  legislation,  and  I  fear  that  this  administration  has  endorsed 
a  bill  that  includes  proposals  to  weaken  the  testing  statutes.  They 
haven't  persuaded  me  that  that  is  in  the  National  interest. 

Thank  you,  Mr.  Chsdrman. 

Senator  Pryor.  Thank  you.  Senator  Roth. 

Our  first  witness  is  the  Honorable  Colleen  Preston,  Deputy 
Under  Secretary  for  Defense  Acquisition  Reform.  We  welcome  you 
this  morning.  I  know  that  you  have  a  statement.  I  have  read  your 
statement,  and  we  would  appreciate  it  if  you  would  summarize 
your  statement  for  the  Committee  this  morning,  if  you  so  desire. 

TESTIMONY  OF  COLLEEN  A.  FRESTON.i  DEPUTY  UNDER  SEC- 
RETARY OF  DEFENSE  (ACQUISITION  REFORM),  DEPART- 
MENT  OF  DEFENSE 

Mrs.  Preston.  Thank  you.  Chairman  Pryor,  Senator  Roth.  I  ap- 
preciate the  opportunity  to  be  here.  I  will  summeirize  as  much  as 
I  can  my  statement,  although  I  do  feel  that  there  are  certain  things 
that  need  to  be  said  so  that  we  can  clear  up  some  misunderstand- 
ings about  what  the  Department's  approach  is  to  operational  test 
and  evaluation. 

First  of  all,  I  would  Uke  to  address  the  ASPJ  issue  which  you 
have  spoken  to  Secretary  Perry  and  Deputy  Secretary  Deutch 
about.  I  would  Uke  to  reaffirm,  on  the  pubUc  record,  that  because 
this  system  did,  in  fact,  fail  its  operational  test  and  evaluation, 
failed  to  satisfy  the  test  criteria,  the  Department  has  made  a  com- 
mitment not  to  purchase  any  further  systems. 

However,  as  you  are  well  aware,  95  of  the  systems  had  already 
been  produced.  There  are  no  plans  within  the  Department  to  pur- 
chase any  additional  ASPJs,  period.  As  you  know,  the  F-14D  re- 
quires a  self-protection  jammer  and  is  configured  only  to  accept  the 
ASPJ  system.  As  part  of  the  F-14D  operational  test  and  evaluation 
plan,  it  is  now  the  DOD  strategy  to  test,  while  they  are  testing  the 
F-14D,  the  ASPJ  as  part  of  the  F-14D  EW  suite. 


1  The  prepared  statement  of  Mrs.  Preston  appears  on  page  36. 


If  it  turns  out  that  the  F-14D  is  more  survivable  with  the  EW 
suite  than  without  it,  then  we  will  deploy  only  those  systems  that 
are  currently  in  our  inventory  on  F-14s.  The  Deputy  Secretary  has 
committed,  however,  that  he  will  instruct  the  Navy  not  to  accom- 
plish any  of  that  testing  until  the  Director  of  Operational  Test  and 
Evaluation  has  been  confirmed,  is  on  the  job,  and  offers  rec- 
ommendations as  to  the  adequacy  and  utility  of  the  tests. 

Senator  Pryor.  May  I  ask  a  question? 

Mrs.  Preston.  Certainly. 

Senator  Pryor.  Are  the  sales  of  the  ASPJ  still  going  forward 
overseas? 

Mrs.  Preston.  Yes,  sir.  In  fact,  I  was  just  going  to  address  that. 

Senator  Pryor.  Before  operational  testing  proved  them  to  be 
worthy? 

Mrs.  Preston.  Yes,  sir.  What  we  have  done  is  said  to  foreign 
customers  that  the  ASPJ  may  be  sold  as  a  commercial  item  by  the 
company  to  a  contractor  or  another  government.  However,  because 
the  system  did  not  pass  operational  testing,  DOD  will  not  allow  a 
purchase  of  the  ASPJ  system  as  an  FMS  ssQe. 

Senator  Pryor.  Do  we  have  to  install  those  systems  in  the  planes 
used  by  our — by  the  way,  who  are  we  selling  to,  our  enemies  or  our 
alhes? 

Mrs.  Preston.  To  our  allies,  sir;  I  beheve  the  Swiss  and  the 
Finns. 

Senator  Pryor.  Do  they  know  that  this  system  has  not  passed 
operational  testing? 

Mrs.  Preston.  Yes,  sir,  they  do. 

Senator  Pryor.  And  they  are  buying  them  anyway? 

Mrs.  Preston.  Yes,  sir.  Now,  I  have  to  tell  you 

Senator  Pryor.  We  have  some  very  good  salesmen,  I  think,  in 
this  country. 

Mrs.  Preston  [continuing.]  — that  we  are  selling  aircraft  to  these 
same  countries.  Those  aircraft  will  have  to  have  the  ASPJ  system 
integrated  if  that  country  decides  to  buy  the  ASPJ.  There  have 
been  several  decisions  made,  on  a  case-by-case  basis,  as  to  whether 
or  not  FMS  funds  may  be  used  to  pay  for  the  integration  of  the 
ASPJ  on  the  aircraft. 

Senator  Pryor.  I  interrupted  you.  If  you  would  like 

Senator  ROTH.  Could  I  ask  one  question,  Mr.  Chairman? 

Senator  Pryor.  Certainly. 

Senator  Roth.  When  you  say  the  foreign  buyer  is  aware  of  the 
situation  with  respect  to  the  ASPJ,  how  are  they  informed?  On 
what  basis  do  we  know  that  the  foreign  buyer  is  fully  acquainted 
with  what  has  proved  to  be  defects  in  the  operational  testing? 

Mrs.  Preston.  I  cannot  tell  you  that.  We  can  provide  an  answer 
for  the  record.  The  mere  fact  that  it  is  not  in  the  U.S.  inventory 
is  typically  the  first  tip-off  to  anyone  that  we  are  not  providing  sup- 
port for  the  system.  Most  FMS  customers  like  to  purchase  items 
that  are  in  the  inventory  that  they  know  we  can  support  with 
spare  parts  because  they  use  our  logistics  system. 
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Korea,  Finland  and  Switzerland  have  all  been  advised  that  the  system  failed  the 
OT&E  tests  and  would  not  be  purchased  by  DOD.  They  were  also  advised  that  for 
this  reason  the  U.S.  would  not  offer  the  system  via  Foreign  Military  Sales  (FMS). 
We  have  not,  nor  do  we  think  it  proper  carried  the  issue  any  further  as  the  matter 
is  now  in  the  commercial  arena.  The  three  countries  have  evaluated  the  situation 
and  the  other  systems  available  commercially  and  all  three  have  decided  to  continue 
to  attempt  to  purchase  the  ASPJ  for  their  mihtary  forces.  Our  position  on  the  mat- 
ter has  to  be  one  of  even-handed  treatment  of  all  U.S.  contractors  and  to  support 
the  wishes  of  our  customers  in  the  absence  of  a  DOD  inventory  system  that  meets 
the  customer's  needs. 

Senator  Roth,  Is  the  Pentagon  taking  the  position,  let  the  buyer 
bew£ire? 

Mrs.  Preston.  That  is  correct.  It  failed  operational  test. 

Senator  RoTH.  Who  are  these  being  sold  to? 

Mrs.  Preston.  I  beUeve  the  Swiss,  the  Finns  and  the  Koreans. 

Senator  Roth.  Now,  these  are  friends  and  allies  with  whom  it  is 
conceivable  we  may  be  in  combat  together.  Do  you  think  that  is  in 
our  interests,  for  example,  in  the  case  of  Korea  for  them  to  acquire 
a  system  that  we  find  not  effective? 

Mrs.  Preston.  I  can't  answer  that  because  I  don't  know  the  spe- 
cifics of  why  they  made  the  decision  to  buy  the  ASPJ. 

Senator  Roth.  But  if  I  understand  what  you  are  saying,  the  Pen- 
tagon sort  of  t£ikes  a  hands-off  position  and  if  somebody  else  wants 
to  buy  them,  that  is  up  to  them,  even  though  we  as  a  Government 
know  otherwise. 

Mrs.  Preston.  But  they  are  provided  the  information  that  these 
systems  did  not  pass  operational  test  and  evaluation.  That  is  why 
we  will  not  sell  them  through  FMS  sales. 

Senator  ROTH.  Could  we  have  for  the  record  a  copy  of  whatever 
information  we  disclose  to  the  foreign  buyer? 

Senator  Pryor.  We  could  make  that  request.  I  don't  know  if  that 
is  confidential  or  not.  If  it  is  not  classified. 

Senator  Roth.  I  obviously  wouldn't  want  it  if  it  is  classified. 

Senator  Pryor.  I  think  that  would  be  £in  appropriate  request. 

May  I  ask,  also,  and  then  we  are  going  to  let  you  continue — we 
apologize — how  much  do  you  assess  that  each  ASPJ  unit  costs  the 
manufacturer? 

INSERT  FOR  THE  RECORD 
AIRBORNE  SELF-PROTECTION  JAMMER  UNIT  COST 

The  average  unit  target  price  for  the  last  limited  production  lot  was  $2.5  milUon 
for  an  F-18  configuration  and  $3.0  million  for  the  F-14  configuration.  The  difference 
is  that  the  F-14  requires  an  additional  receiver  and  transmitter  for  adequate  cov- 
erage. 

Mrs.  Preston.  I  don't  know  the  answer  to  that. 

Senator  Pryor.  Our  figures  show  about  $3  miUion.  Now,  how 
much  are  we  selling  these  units  for? 

Mrs.  Preston.  I  can't  answer  that  either.  I  would  be  happy  to 
do  that  for  the  record. 


INSERT  FOR  THE  RECORD 
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The  U.S.  Government  is  not  selling  Airborne  Self-Protection  Jammer  systems. 

Senator  Pryor.  Are  we  getting  more  or  less  than  $3  million 
each? 

Mrs.  Preston.  I  have  no  idea.  They  are  commercial  sales  and  I 
would  have  to  go  back  to  the  contractor. 

Senator  Pryor.  They  are  commercial  sales,  but  they  were  pro- 
duced with  taxpayers'  dollars,  right? 

Mrs.  Preston,  No,  sir.  The  ones  we  purchased  were  paid  for  with 
taxpayer  dollars.  The  ones  that  are  being  sold,  if  they  are  sold  out 
of  inventory,  would  be  paid  for  by  the  purchasing  country,  and  the 
U.S.  Government  would  be  reimbused. 

Senator  Pryor.  Where  else  would  they  be  coming  from?  You  are 
not  producing  any  more,  are  you,  right  now?  Where  else  would  they 
come  from  except  from  out  of  inventory? 

Mrs.  Preston.  I  can't  answer  that. 

Senator  Pryor.  OK.  Continue,  please. 

Mrs.  Preston.  In  terms  of  the  operational  test  and  evaluation  or- 
ganization, the  Department  is  strongly  committed  to  the  independ- 
ence of  the  Director  of  Operational  Test  and  Evaluation  and  to  the 
live-fire  testing  program.  The  President's  choice  of  the  new  Director 
of  Operational  Test  and  Evaluation  is  at  this  time  going  through 
the  Administration's  review  and  clearance  process. 

In  addition,  although  Mr.  Deutch  firmly  believed  that  the  Direc- 
tor of  Operational  Test  and  Evaluation  should  work  hand-in-glove 
with  the  Under  Secretary  of  Defense  for  Acquisition  and  Tech- 
nology (USD(A&T)),  he  has  just  recently  signed  a  memorandum 
making  clear  that  the  Director  of  OT&E  reports  solely  and  directly 
to  the  Secretary  and  Deputy  Secretary  of  Defense,  and  that  there 
is  a  close  working  relationship,  what  we  call  a  dotted  line  relation- 
ship, to  the  USD(A&T)  and  the  rest  of  the  acquisition  community. 
So  I  think  that  answers  the  question  about  where  the  Director  of 
Operational  Test  and  Evaluation  will  be  reporting. 

With  respect  to  the  time  that  it  has  taken  to  announce  an  ap- 
pointment of  the  Director  of  Operational  Test  and  Evaluation,  I 
might  also  indicate  that  we  have,  in  fact,  no  current  appointee  as 
Service  Acquisition  Executive  in  either  the  Air  Force  or  the  Army. 
There  are  a  number  of  appointments  that  have  not  been  made  in 
the  Department  of  Defense.  We  are  working  as  hard  as  we  can  to 
get  these  individuals  appointed  because  for  every  person  we  are 
missing,  the  rest  of  us  are  doing  double  duty  and  I  can  assure  you 
it  is  not  fun.  But  it  takes  a  long  time  to  get  through  the  process 
and  we  are  working  on  it. 

If  you  would  like  me  to  address  the  chart,  I  would  be  happy  to 
do  that. 

Senator  Pryor.  Well,  last  year,  I  think,  in  the  1994  DOD  Author- 
ization bill,  language  was  included  that  the  OT&E  office  would  re- 
port not  any  longer  directly  to  the  Secretary  of  Defense,  but  to  the 
Comptroller.  Is  that  correct? 

Mrs.  Preston.  That  is  correct.  There  were  some  proposals  that 
were  made  and  there  is  language  that  would  authorize  him  to  re- 
port through  the  Comptroller. 
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ing"? 


Senator  Pryor.  Now,  what  is  the  Department  of  Defense  propos- 


Mrs.  Preston.  The  Department  is  not  proposing  anything  at  this 
point.  The  decision  has  been  made,  and  the  decision  is  that  the  Di- 
rector of  Operational  Test  and  Evaluation  will  report  directly— 
where  that  dotted  line  now  stands  is  a  soUd  Une.  The  position  does 
not  exist,  as  shown  on  this  chart,  under  the  Under  Secretary  of  De- 
fense for  Acquisition  and  Technology.  Instead  there  is  a  "dotted 
Une"  to  the  USD(A&T). 

Senator  Pryor.  Who  is  the  director  going  to  report  to? 

Mrs.  Preston.  The  Director  is  going  to  report  to  the  Secretary 
and  Deputy  Secretary. 

Senator  Pryor.  Directly? 

Mrs.  Preston.  Directly. 

Senator  Pryor.  No  middle  people,  no  middleman? 

Mrs.  Preston.  No  middle  people,  no  one  in  the  middle,  period. 
They  heard  your  concerns.  The  discussions  at  the  point  in  time 
when  the  chart  you  have  depicted  was  created  were  early  discus- 
sions on  organizational  structures.  In  response  to  the  concerns  that 
have  been  raised  primarily  by  you,  Mr.  Chairman,  Senator  Roth 
and  others,  the  decision  was  made  to  have  the  Director  report  to 
the  Secretary  of  Defense  and  the  Deputy  Secretary. 

Senator  Pryor.  Please  continue. 

Mrs.  Preston.  You  also  asked  that  we  address  the  issue  of  sim- 
ulation. I  am  just  going  to  touch  on  that  briefly.  Simulation  is  obvi- 
ously a  technology  that  the  Department  will  increasingly  be  using 
to  reduce  both  the  cost  £uid  time  of  development,  as  well  as  test 
and  evaluation. 

Simulation  will  be  used  to  support,  not  to  replace,  realistic  oper- 
ational testing.  I  think  that  there  are  a  number  of  advantages  that 
simulation  provides  us.  The  technology  has  improved  dramatically 
over  the  years  in  allowing  us  to  use  technology  to  begin  testing 
very  early  on  in  the  process — something  that  you  mentioned.  Sen- 
ator Roth,  as  critical. 

At  that  point  in  time,  very  early  on  in  the  testing  process,  sim- 
ulation is  often  the  only  thing  we  can  do  because  we  don't  have  a 
system  to  test.  At  that  point,  we  can— and  the  Army  has — do  some 
wonderftil  things.  I  am  sure  the  Army  would  be  happy  to  have  you 
walk  through  their  simulation  and  modeling  techniques  with  them. 
They  can  through  simulation  look  at  a  system  configured  for  com- 
bat, actually  exercise  it  in  that  environment  and  see  whether  or  not 
the  system  works  as  it  was  intended,  long  before  we  build  it. 

Now,  because  I  think  it  is  something  that  is  very  critical,  I  would 
Uke  to  turn  to  what  we  are  proposing  with  respect  to  reform  of  the 
operational  and  live-fire  testing  statutes.  As  I  indicated  previously, 
the  Secretary  and  Deputy  Secretary  of  Defense  beUeve  that  inde- 
pendent operational  and  Uve-fire  test  and  evaluation  is  a  critical 
part  of  the  acquisition  process.  I  share  that  beUef,  and  I  am/ the 
person  charged  with  making  sure  that  acquisition  reform  is  carried 
out  throughout  the  Department  of  Defense.  I  can  assure  you  that 
there  is  no  intention  on  our  part  whatsoever  of  weakening  and,  in 
fact,  we  would  like  to  strengthen  the  operational  test  and  evalua- 
tion statutes.  The  reason  is  simple. 
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Senator  Pryor.  I  am  going  to  have  to  stop  you  there  just  a 
minute.  I  apologize.  Be  really  careful  what  you  say  about  this  be- 
cause I  truly  mean  this  and  I  say  it  with  all  sincerity.  Don't  insult 
us  and  don't  insult  our  intelligence  by  saying  that  you  are  trying 
to  strengthen  the  operational  testing  office  by  supporting  these 
amendments  that  weaken  this  department. 

Are  you  supportive  of  these  amendments  in  S.  1587? 

Mrs.  Preston.  I  would  like  to  explain  to  you  where  we  were  in 
the  process  and  where  we  are  now  in  terms  of  the  amendments  we 
are  supporting  because  our  position  has  changed. 

Senator  Pryor.  Well,  it  is  section  3013,  so  people  will  know.  Be- 
ginning on  line  23,  page  160:  "The  Secretary  of  Defense  may,  for 
a  particular  major  defense  acquisition  program,  prescribe  and 
apply  different  operational  test  and  evaluation  procedures  than 
those  prescribed  under  other  subsections."  Here  is  the  key  lan- 
guage: "The  Secretary  may  transmit  or  may  waive  operational  test- 
ing if  he  finds  it  to  be  unreasonably  expensive,  impractical,  cause 
unwarranted  delay,  or  be  unnecessary  because  of  the  acquisition 
strategy  for  that  system."  The  acquisition  strategy  once  again  is 
driving  these  purchases.  We  call  this  the  Mack  truck  amendments. 
The  Mack  truck  amendments  create  a  loophole  big  enough  to  drive 
a  Mack  truck  through.  I  hope  you  don't  support  this. 

Mrs.  Preston.  I  would  like  to  explain  to  you,  if  I  could,  in  detail, 
what  it  is  we  are  supporting,  and  why. 

Senator  Pryor.  Please. 

Mrs.  Preston.  There  is  certainly  room  for  disagreement,  and  I 
would  never  say  anything  that  would,  or  I  hope  could,  be  taken  to 
insult  your  intelligence,  because  I  know  that  you  are  the  expert  on 
this  issue.  You  have  been  involved  in  this  issue  for  many,  many 

years. 

We  are  trying  to  do  some  things  to  accommodate  the  interests  of 
streamlining  the  acquisition  process,  making  sure  that  we  have  ac- 
countability and  responsibility  in  the  same  individuals,  and  pursu- 
ing various  other  initiatives.  What  we  are  doing  is  saying  that  we 
need  to  balance  the  interests  here.  That  is  one  of  the  reasons  that 
we  are  proposing  some  changes. 

It  is  in  our  best  interest  to  do  that  because  as  the  cost  of  new 
weapons  systems  increases  and  the  DOD  budget  decreases,  we 
can't  afford  to  field  a  system  that  doesn't  meet  operational  needs 
and  is  not  operationally  suitable  and  effective. 

I  would  also  indicate  to  you  that  you  had  a  previous  chart  up 
here  on  some  systems  where  testing  had  not  been  accomplished.  I 
hate  to  get  into  pointing  fingers  in  terms  of  Administrations,  but 
every  one  of  those  decisions  on  those  systems  was  made  before  this 
Administration  took  office.  If  I  am  not  mistaken,  every  one  of  the 
decisions  on  each  of  those  systems  was  made  long  before  the  cur- 
rent Secretary  of  Defense  or  Deputy  Secretary  of  Defense  came  into 
their  positions. 

Senator  Roth.  Well,  I  don't  want  to  see  poUtics  get  into  it,  but 
I  would  just  suggest  what  has  happened  in  the  last  14  months  that 
shows  any  change  of  policy.  I  mean,  a  certification  that  such  test- 
ing would  be  unreasonably  expensive,  impractical,  cause  unwar- 
ranted delay  or  be  unnecessary  because  of  the  acquisition  strategy 
for  the   system — and  the  administration  supports  that — frankly, 
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that  exception  is  so  broad-based,  such  an  open-door  policy,  that  it 
makes  a  mockery  of  operational  testing. 

I  might  be  able  to  buy  some  kind  of  waiver  if  I  ever  became  per- 
suaded, and  I  don't  care  what  administration  it  is,  that  they  were 
really  strictly  enforcing  fly  before  you  buy.  There  may  be  situations 
where,  it  might  make  some  sense,  but  the  fact  is  that  without  this 
language,  the  Pentagon  has  found  ways  and  means  to  avoid  it. 

When  you  talk  about  that  chart  that  we  had  before  and  the  dot- 
ted lines,  it  isn't  whether  it  is  a  straight  line  or  a  dotted  line;  it 
is  what  the  practice  in  the  Pentagon  itself  is.  What  people  tell  us 
is  that,  in  fact,  on  a  day-to-day  basis  the  operational  testing  direc- 
tor is  going  to  be  reporting  at  a  lower  level,  and  that  bothers  me. 

Mrs.  Preston.  Sir,  that  is  not  correct.  The  Secretary's  memo  was 
signed  out  on  March  17.  There  may  be  some  people  who  have  not 
been  made  aware  of  that  decision  within  the  Department  of  De- 
fense, but  I  find  it  hard  to  believe  that  the  Secretary  and  Deputy 
Secretary  would  be  willing  to  commit  that  to  you  and  then  basi- 
cally, in  essence,  lie  to  you. 

Senator  Roth.  Let  me  say — and  I  don't  want  to  continue  because 
we  want  to  hear  what  else  you  have  to  say — it  is  not  what  anybody 
says  that  is  going  to  be  important.  It  is  going  to  be  6  months  from 
now,  or  whenever  we  find  out  what  the  true  practice  is.  That  is 
what  we  are  going  to  judge  it  by. 

Mrs.  Preston.  Great.  We  wouldn't  disagree  with  that  at  all. 

Senator  Roth.  Good. 

Mrs.  Preston.  As  you  know,  the  DOD  Advisory  Panel  on 
Streamlining  and  Codifying  Acquisition  Laws,  what  we  call  the 
Section  800  Panel,  examined  the  three  current  general  testing  stat- 
utes, 10  U.S.C.  2366,  2399  and  2400.  They  found  that  the  statutes 
promote  unnecessary  oversight,  have  excessive  definitional  and  re- 
porting detail,  do  not  allow  the  Department  to  exercise  sound  man- 
agerial discretion  in  testing  appUcation  and  implementation,  and 
limit  the  participation  of  systems  contractors  in  operational  tests 
where  such  involvement  is  necessary  and,  in  fact,  prudent. 

The  panel  recommended  the  adoption  of  a  streamlined  testing 
statute  that  set  forth  the  basic  rule  that  vuhierability  or  lethality 
and  operational  testing  should  occur  before  proceeding  beyond  low- 
rate  initial  production  (LRIP).  The  proposed  statute  then  vested 
the  discretion  in  the  Secretary  to  implement  the  required  testing 
statutes  and  provided  broad  guidelines. 

The  panel  also  recommended  an  alternative,  and  that  was 
amendments  to  each  of  the  specific  statutes  themselves.  When  con- 
sidered by  the  Acquisition  Reform  Senior  Steering  Group,  which  in- 
cludes representatives  of  each  of  the  Services  and  the  testing  com- 
munity both  within  the  Services  and  within  OSD,  the  decision  was 
made  to  recommend  to  Mr.  Deutch  at  that  time  that  we  proceed 
with  the  Section  800  Panel  recommendation  for  total  streamlining. 
It  was  felt  that  the  statutory  language  was,  in  fact,  inflexible  and 
that  it  did  not  add  to  the  body  of  regulatory  provisions  that  are 
currently  now  in  place  as  a  result  of  the  legislation. 

The  legislation  was  necessary  to  force  the  Department  to  imple- 
ment what  you  wanted.  That  has  now  been  done  through  regula- 
tion and  the  decision  was  that  it  did  not  need  to  remain  in  legisla- 
tion. 
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Senator  Pryor.  Was  that  a  unanimous  decision? 
Mrs.  Preston.  No,  and,  in  fact,  there  were  two  people  who  op- 
posed that.  They  were  the  Director  of  Operational  Test  and  Evalua- 
tion and  the  DOD  Inspector  General. 

Senator  Pryor.  Did  not  their  recommendations  carry  any 
weight? 

Mrs.  Preston.  Yes,  sir,  they  carry  a  lot  of  weight.  In  fact,  Mr. 
Deutch  met  privately  with  the  Inspector  Greneral.  The  comments  of 
the  Director  of  Operational  Test  and  Evaluation  were  forwarded  to 
Mr.  Deutch.  He  read  them.  He  read  the  responses  that  were  pro- 
vided by  the  Steering  Group  and  decided  to  go  forward  with  this 
recommendation  at  that  time. 

With  respect  to  the  legislation  you  have  before  you,  S.  1587,  and 
the  various  other  legislative  proposals,  none  of  them  come  close  to 
adopting  either  the  streamhned  statute  proposed  by  the  Section 
800  Panel  or  the  actual  alternative  that  the  Section  800  Panel  rec- 
ommended. S.  1587  comes  the  closest  to  adopting  the  Section  800 
alternative  for  live-fire  testing  and  low-rate  initial  production,  but 
it  does  not  make  the  changes  to  Operational  Test  and  Evaluation 
that  we  feel  are  necessary  to  provide  us  some  flexibility,  and  pro- 
vides no  relief  for  contractor  support. 

Senator  Pryor.  May  I  ask  this  question?  Do  we  have  an  Inspec- 
tor General  for  the  Department  of  Defense? 

Mrs.  Preston.  No,  sir,  we  do  not.  That  is  one  of  the  appoint- 
ments that  is  still  being  held  up. 
Senator  Pryor.  How  long  has  that  been  vacant? 
Mrs.  Preston.  Since  the  beginning  of  the  Administration. 
Senator  Pryor.  So  we  don't  have  an  Inspector  General  and  we 
don't  have  a  Director  of  the  Office  of  Testing.  Who  did  Mr.  Deutch 
meet  with? 

Mrs.  Preston.  Mr.  Deutch  met  with  the  Acting  Inspector  Gen- 
eral, Mr.  Derek  Vander  Schaaf. 

Senator  Pryor.  What  sort  of  a  signal  is  sent  out  when  we  have 
these  vacancies  for  this  length  of  time  in  very,  very  critical  func- 
tions of  the  Department?  What  sort  of  a  signal? 

Mrs.  Preston.  Sir,  I  can  only  tell  you  that  we  wish  that  every 
one  of  those  positions  was  filled.  I  know  that  the  Secretary  and 
Deputy  Secretary  would  like  nothing  better  than  to  have  every  sin- 
gle political  appointment  finished.  There  is  a  process  that  the  Ad- 
ministration goes  through.  It  involves  the  FBI,  as  you  know,  and 
there  is  only  so  much  that  can  be  done  to  push  that  process  faster 
than  it  is  going.  It  is  not  for  lack  of  recommendations  from  the  Sec- 
retary. 

Getting  down  to  specifics,  as  I  have  mentioned,  I  believe  that 
there  are  four  problems  with  the  current  statutes  that  must  be 
solved.  You  noted  some  of  these  objections  or  concerns  with  the  cur- 
rent statutes.  First,  the  laws  are  inflexible  and  in  some  cases  unre- 
alistic, causing  us  to  expend  a  lot  of  money  for  a  little  gain.  One 
example  of  that  would  be  with  the  MlAl  live-fire  test  program 
which  required  the  firing  of  48  shots  against  five  combat  loaded 
Abrams  tanks.  The  approximate  cost  of  the  tests  was  about  $27 
million,  $12  million  for  the  test  program  and  $15  million  for  the 
tanks  that  were  destroyed  or  damaged  in  the  test. 
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We  believe  in  a  more  systematic  approach  to  testing — and,  again, 
some  of  this  is  the  Department  being  conservative  and  misinter- 
preting what  the  statute  requires  in  some  cases.  But  if  we  had  had 
the  opportunity  to  set  out  a  testing  plan  that  took  advantage  of 
testing  components,  subsystems,  et  cetera,  early  on  and  then  made 
minimal  use  of  a  full-up  system  configured  for  combat,  we  could 
have  saved  an  enormous  amount  of  money,  and  I  think  that  that 
is  in  the  best  interest  of  the  taxpayer. 

Senator  Pryor.  Is  your  whole  argument  here  that  operational 
testing  is  too  expensive? 

Mrs.  Preston.  No,  sir.  I  am  saying  that  the  way  it  is  applied 
right  now,  it  is  in  some  cases  an  expense  that  is  being  incurred 
without  any  benefit. 

Senator  Pryor.  All  right.  Let  me  direct  you  to  this  blue  and 
white  chart,  "Is  Operational  Testing  Too  Expensive?"  i  Now,  you 
have  got  to  have  real  good  eyes,  better  than  mine.  Down  at  the  bot- 
tom there  is  a  little  red  barometer.  Senator  Roth,  a  little  red  line. 
That  represents  the  cost  of  operational  testing  for  the  M1A2  tank. 
You  can  see  that  .8  percent  went  to  operational  testing  for  that 
tank.  The  testing  for  the  ASP  J  Jammer  was  only  .2  percent  of  its 
total  cost,  and  the  C-17  and  the  B-IB,  you  see,  cost  less  than  one- 
tenth  of  1  percent. 

So  I  think  we  are  looking  at  operational  testing  as  almost  a  non- 
existent cost  when  you  look  at  the  total  expenditure  of  dollars  here. 
It  is  a  very,  very  small  fi*action. 

Mrs.  Preston.  However,  if  I  am  not  mistaken,  the  systems  that 
you  have  up  there  that  you  have  costed  out  were  systems  that  you 
also  pointed  out  did  not  go  through  operational  test  or  went 
through  only  a  minor  part  of  the  operational  test. 

Senator  Pryor.  Staff  advises  me  that  they  had  already  procured 
these  weapons  before  operational  testing  occurred. 

Mrs.  Preston.  Right.  I  am  not  trying  to  argue  with  you,  Senator. 
I  am  just  trying  to  indicate  that  that  is  not  indicative  of  the  true 
cost  of  £in  operational  test  program,  if  one  is  accomplished,  because 
in  none  of  those  instances  was  a  true  operational  test  accom- 
plished. 

Senator  Pryor.  Now,  you  know,  testing  is  not  a  very — I  hate  to 
use  the  word  "sexy,"  but  it  is  not  a  very  high  priority  item  it  seems 
like  today.  There  are  not  a  lot  of  people  interested  in  this,  except 
the  people  in  the  battlefield.  They  are  going  to  be  interested  in  the 
outcome  of  all  of  this,  and  they  are  certainly  going  to  be  interested 
in  the  outcome  of  S.  1587.  They  will  also  be  interested  in  whether 
this,  as  we  call  it.  Mack  truck  simendment,  remains  in  the  bill. 

Making  certain  that  these  weapons  work  is  a  very,  very  impor- 
t£int  part  of  our  military  preparedness.  Whoever  our  enemies  might 
be  today,  we  had  better  believe  that  their  intelligence  knows  if 
these  weapons  work  or  don't  work,  and  I  think  that  we  have  got 
to  keep  that  in  mind  throughout  this  whole  process. 

If  you  would  please  continue. 

Mrs.  Preston.  Let  me  quickly  go  through  some  of  the  other 
points  and  then  get  to  what  is  really,  I  think,  important  to  you,  and 
that  is — what  are  we  proposing  with  respect  to  changes  now. 
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One  other  example  of  a  point  that  we  have  brought  out  is  that 
the  current  laws  artificially  require  segregation  of  developmental 
and  operational  tests.  For  example,  on  the  F-22,  there  was  a  total 
7,000-hour  test  program.  Of  that  test  program,  only  540  were  nec- 
essary for  operational  test. 

Although  it  is  difficult  to  make  an  assessment  looking  in  a  vacu- 
um, if  the  developmental  and  operational  test  communities  were 
able  to  work  closer  together  and  take  advantage  of  the  data  that 
is  developed  in  a  more  synergistic  manner,  it  would  obviously  bene- 
fit both  because  artificially  segregating  the  data  reduces  the  useful- 
ness of  the  data  that  is  developed  by  either  of  the  parties  and 
forces  judgments  to  be  made  on  a  small  part  of  the  data  that  is  ac- 
tually accumulated  during  an  entire  test  program. 

Third,  we  are  trying  as  part  of  our  acquisition  reform  initiatives 
to  establish  accountability  and  responsibility  within  the  same  indi- 
viduals. So,  for  example,  where  we  have  responsibility  for  review 
of  a  system  at  the  OSD  level,  we  would  like  to  see  testing  reviewed 
at  the  OSD  level.  However,  if  there  has  been  a  delegation  of  that 
authority  to  the  Service  Secretary  and  the  Service  Secretary  is  the 
one  who  is  making  all  of  the  milestone  decisions  on  proceeding  with 
the  program,  then  we  believe  the  testing  organization  at  that  level 
should  make  the  decisions  with  respect  to  testing.  So  as  a  result, 
we  are  asking  that  there  be  a  match-up  of  the  milestone  decision- 
making authorities  with  the  testing  authority  approval  levels. 

Then,  finally,  just  to  mention  the  issue  of  the  Services  using  con- 
tractor support,  we  have  some  proposals  on  this  that  we  think 
would  clarify  the  issue  of  when  contractor  support  can  or  cannot  be 
used.  We  had  an  example  on  the  AMRAMl  system  where  a  sub- 
contractor for  the  contractor  hired  to  do  the  operational  test  had 
been  doing  the  developmental  test.  When  he  was  hired  to  be  the 
subcontractor  on  the  operational  test,  they  had  to  stop  the  oper- 
ational testing  until  they  could  resolve  the  situation  of  making  sure 
that  the  individual  who  had  the  data  that  they  felt  they  needed  to 
finish  conducting  the  tests  was  available. 

Because  of  the  prohibition  in  the  statute,  which  they  thought 
they  weren't  able  to  work  out,  what  ended  up  happening  is  that 
this  data  analyst  actually  resigned  from  the  company,  set  up  his 
own  corporation,  and  then  he  subcontracted  with  Northrop,  but  we 
lost  3  to  4  months  while  that  was  going  on. 

Again,  another  example  on  the  F-22 — 

Senator  Pryor.  Were  these  contracts  sole-source  contracts? 

Mrs.  Preston.  I  can't  answer  that.  I  will  provide  you  an  answer 
for  the  record. 

Senator  Pryor.  I  will  bet  you  they  were. 

INSERT  FOR  THE  RECORD 

COMPETITIVE  CONTRACT 

The  contract  for  operational  test  support  was  competed  by  the  Department  of  De- 
fense. The  data  anafyst  was  hired  by  the  support  contractor  as  a  subcontractor  con- 
svdtant  without  the  benefit  of  competition. 

Senator  Pryor.  Please  go  forward. 

Mrs.  Preston.  OK.  The  second  situation  is  the  situation  we  en- 
counter with  the  so-called  contract  advisory  and  assistance  services 
(CAAS)    contractors.    Actually,    that    was    an    example    of   the 
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AMRAAM  contract  advisory  and  assistance  services  problem  where 
you  have  a  contractor  not  involved  in  either  development,  with  the 
exclusion  of  testing,  or  production  of  the  system.  Is  there  a  rational 
reason  to  say  that  you  can't  be  involved  in  both  developmental  and 
operational  testing? 

Another  situation  is  where  you  have  a  contractor  who  is,  in  fact, 
the  one  who  is  developing  and  potentially  producing  the  system.  In 
this  case  of  the  F-22,  the  contractor  had  a  full  mission  simulator 
that  was  essential  to  evaluate  the  F-22  against  advanced  threats 
and  in  a  dense  environment  that  cannot  be  tested  in  the  field.  That 
FMS  was  the  only  existing  simulator  that  could  satisfy  that  F-22 
requirement.  It  was  located  in  the  systems  contractor's  facility  and 
operated  by  the  systems  contractor. 

For  us  to  go  in  and  duplicate  that  either  by  buying  a  machine 
or  by  hiring  and  training  personnel  that  would  do  that  test  for  the 
Department  doesn't  make  any  sense.  What  we  want  to  preclude  is 
that  contractor  from  being  involved  in  setting  the  test  parameters 
or  from  doing  any  determination  of  whether  or  not  the  system  met 
the  testing  requirements.  •      .     i     ^ 

What  are  we  proposing?  We  have  spent  a  lot  of  time  m  the  last 
several  weeks  trying  to  see  if  we  could  reach  an  accommodation  in 
terms  of  responding  to  your  concerns  and  the  testing  communities 
within  the  Department.  I  can't  say  that  there  is  agreement  because 
there  isn't.  If  there  were,  you  would  probably  be  suspicious  of  what- 
ever we  agreed  to  anyway,  or  at  least  wonder  what  we  did  to  the 
operation£d  test  people  in  DOD  to  get  them  to  agree  to  this. 

But,  basically,  we  are  proposing,  with  respect  to  Uve-fire  testing, 
to  insert  a  clause  that  would  require  that  testing  be  accomplished 
first  at  the  component,  subsystem  and  sub-assembly  level,  cul- 
minating with  tests — and  this  would  be  a  clarification  of  the  exist- 
ing statutes — of  the  complete  system  configured  for  combat. 

So  we  are  doing  exactly  what  you  wanted,  Senator  Roth,  which 
is  emphasizing  testing  early  in  development — and  this  is  actually 
a  recommendation  from  the  live  fire  testing  community.  We  can't 
take  credit  for  this.  This  is  something  they  proposed,  and  rightly 
so,  the  emphasis  on  doing  graduated  testing  as  you  go  along  in  the 
program,  which  will  allow  us  to  do  testing  much  earlier  in  the  proc- 
ess, and  then  culminate  with  the  complete  system  configured  for 
combat. 

Then  we  would  authorize  the  Secretary  of  Defense — and  that  is 
delegable  under  the  current  statute  and  would  remain  delegable  in 
our  proposal  so  that  it  would  go  the  milestone  decision  authority — 
to  waive  the  vulnerability  or  lethality  tests  on  a  complete  system 
configured  for  combat  if  the  Secretary  determines— and  this  again 
is  the  existing  statute — that  it  would  be  unreasonably  expensive 
and  impractical,  and  that  testing  of  components,  subsystems  and 
assemblies,  or  realistic  replicas,  et  cetera,  would  provide  an  accept- 
able assessment  of  the  vulnerability  of  the  system  as  configured  for 
combat. 

Now,  again,  that  is  existing  law.  What  we  are  adding  is  the  clari- 
fying words  that  there  be  testing  on  a  complete  system  configured 
for  combat.  Those  clarifying  words  are  right  now  in  the  definitional 
section  of  the  current  statute,  and  we  believe  it  is  a  little  confusing, 
and  that  is  the  only  reason  we  are  proposing  this. 
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Then  we  would  require  that  the  waiver  to  testing,  if  there  is  to 
be  a  waiver  of  testing  of  the  complete  system  configured  for  com- 
bat, must  be  executed  prior  to  engineering  and  manufacturing  de- 
velopment (EMD),  or  in  the  case  of  a  system  for  which  there  is  no 
EMD  phase — we  are  bringing  in  a  lot  of  systems  right  now  that  we 
are  bringing  in  at  Milestone  III  or  Milestone  IV  because  they  are 
commercial  systems,  so  we  don't  have  any  EMD  phase  for  some  of 
these  programs. 

We  are  potentially  going  to  have  a  new  commercial  off-the-shelf 
aircraft  in  the  near  future.  If  we  do  that,  we  are  not  going  to  have 
an  engineering  and  manufacturing  development  phase.  So  we  need 
the  flexibihty  in  the  statute  to  authorize  a  waiver  of  full-up  live- 
fire  testing  after  EMD.  Right  now,  the  statute  does  not  provide  any 
waivers  at  all. 

Senator  Pryor.  Mrs.  Preston,  we  do  have  two  more  witnesses. 
We  have  taken  a  lot  of  your  time  because  we  have  interrupted  you 
several  times,  but  if  you  could  conclude  in  a  concise  manner,  we 
would  appreciate  it.  ,        r   i. 

Mrs.  Preston.  OK.  What  we  would  then  provide  is  that  if  the 
waiver  is  not  executed  in  the  time  limits  that  are  required,  the  Sec- 
retary of  Defense,  without  delegation— which  again  would  be  a 
strengthening  of  the  current  statute— would  have  to  approve  the 
waiver  and  provide  a  report  to  Congress  indicating  that  the  waiver 
occurred. 

With  respect  to  operational  testing,  we  are  proposing  that  we 
align  the  decisionmaking  authorities  with  respect  to  the  testing  pa- 
rameters with  the  milestone  decision  authorities.  So  if  it  is  the 
Service  that  makes  the  decisions  as  to  whether  or  not  to  go  forward 
with  the  program,  it  would  be  the  testing  organization  within  that 
Service  who  would  make  the  testing  decisions,  as  opposed  to  elevat- 
ing it  to  OSD  level. 

Finally,  we  would  allow  the  Secretary  of  Defense  to  authorize  the 
use  of  employees  of  a  contractor  whose  svstem  is  being  tested,  but 
to  provide  analytic  and  logistic  support,  but  only  that  support  and 
only  when  contractor  impartiality  is  assured.  And  we  would  allow 
a  contractor  to  support  both  developmental  and  operational  test 
analysis,  again  only  when  impartiality  is  ensured.  In  neither  case 
would  the  Secretary  be  authorized  to  use  a  contractor  employee  to 
assist  in  the  establishment  of  criteria  for  data  collection,  criteria 
for  periformance  assessment,  or  criteria  relating  to  the  evaluation 
activities. 

If  I  could  just  summarize  then,  Mr.  Chairman,  just  as  in  the 
other  areas  that  we  have  targeted  for  acquisition  reform,  we  need 
to  take  a  good,  hard  look  in  this  area— and  we  think  we  have  done 
that— at  what  we  intend  to  accomplish  with  the  testing  and  evalua- 
tion process  as  it  exists  today;  look  at  re-engineering  that  process 
to  make  it  as  efficient  and  cost-effective  as  it  can  be,  given  the 
changes  in  the  environment  that  have  occurred  since  the  statute 
was  adopted.  . 

We  would  like  to  do  that  while  we  are  protecting  the  public  s  in- 
terest and  protecting  the  intent  of  the  statute.  I  truly  beheve,  and 
you  may  disagree,  but  I  truly  believe  that  the  changes  we  are  pro- 
posing will  do  that.  They  will  accompHsh  the  end  result  that  you 
want.  I  oiJy  ask  that  you  give  us  a  chance  to  do  this,  to  try  this. 
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If  we  don't  perform,  you  always  have  the  opportunity  to  come  back 
in  and  reimpose  statutory  provisions.  You  will  have  the  ammuni- 
tion then  to  do  it.  You  will  have  the  ammimition  with  respect  to 
this  team  in  the  Department  of  Defense  now,  and  again  we  would 
simply  ask  that  we  be  given  the  opportimity  to  try  and  make  some 
efforts  to  ameud  the  statutes  in  a  way  that  we  think  will  make  this 
approach  more  feasible. 

Senator  Pryor.  I  have  two  quick  questions.  If  these  changes 
strengthen  the  operational  testing  statute,  then  why  do  we  find  the 
Office  of  OT&E  not  supporting  these  changes? 

Mrs.  Preston.  There  are  some  changes  that  the  office  actually 
supports  and  some  that  they  don't  support.  In  fact,  I  am  trying  to 
think  of  which  ones  the  operational  test  community  does  not  sup- 
port. We  were  just  having  discussions  as  late  as  last  night,  and 
they  have  not  seen  the  actual  language.  We  are  having  it  drafted 
up  right  now,  and  in  all  fairness  to  them  they  ought  to  be  given 
the  opportunity  to  see  the  language  as  it  is  written  as  opposed  to 
(hscussing  it  verbally  before  they  make  a  decision. 

Senator  Pryor.  I  am  not  going  to  ask  you  to  reveal  the  individ- 
ual whom  the  administration  is  going  to  nominate  to  be  our  testing 
czar.  I  do  not  know  it  myself  yet,  but  has  that  person  been  con- 
fronted with  the  issue  of  S.  1587?  Has  that  person  been  instructed 
on  how  to  answer  questions  regarding  what  we  label  as  the  Mack 
truck  amendments,  which  we  think  weaken  operational  testing? 

Mrs.  Preston.  I  have  never  had  any  discussions  with  the  indi- 
vidual, so  I  can't  say  that  no  one  in  the  Department  has  ever 
talked  to  him  about  these  provisions.  But  the  individual  is  not  re- 
ceiving briefings  at  this  time  £ind  wiU  not  until 

Senator  Pryor.  Does  this  mean  that  we  do  not  know  how  he 
feels  about  these  amendments  that  are  going  to,  in  my  opinion, 
weaken  operational  testing? 

Mrs.  Preston.  I  would  say  that  it  is  a  safe  bet  that  there  isn't 
anyone  in  the  building,  outside  of  about  four  or  five  people,  who 
even  know  what  amendments  we  are  proposing  because  we 

Senator  Pryor.  We  all  know  the  amendments.  Here  is  the  lan- 
guage right  here. 

Mrs.  Preston.  You  are  saying  S.  1587? 

Senator  Pryor.  Yes. 

Mrs.  Preston.  I  have  no  idea  whether  or  not  he  has  any  knowl- 
edge that  S.  1587  exists. 

Senator  Pryor.  Well,  I  think  he  ought  to  know.  I  think  Senator 
Roth  probably  feels  Uke  he  ought  to  know.  This  is  going  to  affect 
his  office,  his  job,  his  challenge. 

Let  me  conclude  by  saying  this,  I  feel  so  strongly  about  this  sec- 
tion, so  strongly,  that  if  the  new  nominee  says  he  supports  this  lan- 
guage, I  reserve  the  right  to  oppose  this  nominee  and  to  try  to  find 
someone  who  believes  that  this  should  not  become  law. 

Mrs.  Preston.  We  will  make  sure  that  once  the  announcement 
of  the  intent  to  nominate  is  made  and  that  individual  has  an  oppor- 
tunity to  come  over  and  the  name  is  actually  revealed  at  that  point, 
that  we  will  communicate  to  him  your  concerns. 

Senator  Pryor.  Senator  Roth? 

Senator  Roth.  Mr.  Chairman,  just  let  me  say  I  have  exactly  the 
same  concerns  and  will  join  you  in  that  effort. 
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Let  me  point  out  that  in  hearings  concerning  the  Federal  Acqui- 
sition Streamhning  Act  of  1993,  the  Defense  Department,  of  course, 
requested  that  there  be  seven  proposed  pilot  programs  and  that  a 
number  of  statutes  be  waived,  and  these  statutes  include  the  oper- 
ational and  live-fire  testing  laws.  Now,  if  the  administration  or  the 
Pentagon  is  supportive  of  operational  testing,  why  do  they  want 
these  waived  in  these  pilot  tests? 

Mrs.  Preston.  Senator,  the  pilot  programs  were  carefully  chosen 
to  demonstrate  that  DOD  could  buy  commercial  off-the-shelf  sys- 
tems to  the  maximum  extent  practical  using  commercial  terms  and 
conditions. 

Senator  Roth.  We  are  supportive  of  that. 

Mrs.  Preston.  OK.  In  many  cases  an  operational  test,  as  re- 
quired by  the  Department  of  Defense  and  required  by  the  statute 
itself,  is  not  necessary.  For  example,  one  of  the  systems  we  are 
buying  is  a  non-developmental  item  (NDI)  aircraft  trainer.  It  is 
going  to  be  tested  to  FAA  standards,  et  cetera,  et  cetera.  It  would 
make  no  sense  for  us  to  go  through  an  operational  test  and  evalua- 
tion that  would  comply  with  the  current  statutes  in  Title  10  for 
that  system. 

Senator  ROTH.  Mr.  Chairman,  our  time  is  running  out.  I  would 
just  like  to  make  one  final  comment,  and  that  is  I  have  this  deep- 
rooted  concern  that  within  the  Pentagon— and  I  don't  care  what  ad- 
ministration it  is — there  is  no  interest  in  operational  testing.  That 
is  the  problem.  I  could  buy  a  lot  of  these  waivers  and  other  things 
if  I  felt  there  was  a  real  policy  of  operational  testing  and  that  they 
were  just  trying  to  find  exceptions  where  there  were  special  rea- 
sons. But,  frankly,  based  on  our  experience  over  the  last  10  years, 
I  see  most  of  these  things  as  means  of  undercutting,  reducing,  if 
not  actually  eliminating  the  concept  of  fly  before  buy. 

Senator  Pryor.  Well,  let  me  ask  Senator  Roth  a  question.  Have 
you  ever  had  anyone  come  to  your  office,  and  lobby  you  on  behalf 
of  operational  testing? 

Senator  Roth.  Never. 

Senator  Pryor.  I  haven't  either.  You  and  I,  I  guess,  are  going  to 
try  to  be  the  lobbyists  for  operational  testing. 

Mrs.  Preston.  Mr.  Chairman,  could  I  respond  to  that  because  I 
think  we  are  trying  to  change  the  way  in  which  DOD  operates?  It 
is  going  to  be  a  cultural  change.  The  entire  acquisition  reform  ef- 
fort is  intended  to  do  that.  We  are  trying  to  make  sure  that  people 
understand  that  we  are  all  part  of  the  same  team. 

I  haven't  been  in  the  Department  very  long,  but  I  know  enough 
about  the  testing  community  to  know  that  we  exist  at  two  ends  of 
the  poles  in  many  cases  with  respect  to  the  acquisition  community 
and  the  testing  community.  There  shouldn't  be  any  reason  for  that, 
in  my  opinion.  The  testing  community  should  be  able  to  be  inde- 
pendent on  their  own  and  resolve  these  differences  with  the  acqui- 
sition community  as  to  what  is  reasonable.  The  problem  is  that  we 
haven't  been  able  to  get  the  pgirties  together  to  agree  on  what  is 
reasonable. 

Senator  Pryor.  We  thank  you.  We  are  going  to  keep  the  record 
open  for  about  5  days  in  case  other  questions  come  in.  We  appre- 
ciate very  much  your  coming  before  the  Subcommittee  this  morn- 
ing. 
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Mrs.  Preston.  Thank  you  very  much. 

Senator  Pryor.  Now,  with  Senator  Roth's  permission,  because  of 
the  time,  I  am  going  to  combine  the  next  two  panels:  Mr.  Lou 
Rodrigues,  Director  of  Systems  Development  and  Production  Is- 
sues, U.S.  Government  Accounting  Office,  and  the  Honorable  Rus- 
sell Murray,  former  Assistant  Secretary  of  Defense. 

We  would  like  for  you  to  both  come  to  the  table,  and  if  you  have 
anybody  that  you  would  like  to  bring  with  you,  feel  free  to  do  so. 
You  have  been  waiting  patiently  and  we  thank  you. 

Mr.  Murray.  It  has  been  a  pleasure  to  listen  to  you.  Senator. 

Senator  Pryor.  Pardon? 

Mr.  Murray.  I  say  it  has  been  a  pleasure  to  listen  to  you. 

Senator  Pryor.  Well,  neither  of  you  are  strangers  to  this  Sub- 
committee. You  have  been  very  helpful,  both  of  you,  and  we  thank 
you  for  coming.  I  believe,  Mr.  Rodrigues,  you  are  up  first  and 
should  you  desire  to  make  a  statement,  we  want  to  hear  it.  If  you 
want  to  put  it  in  the  record,  you  may  summarize  your  statement 
for  us. 

TESTIMONY  OF  LOUIS  J.  RODRIGUES,i  DIRECTOR,  SYSTEMS 
DEVELOPMENT  AND  PRODUCTION  ISSUES,  NATIONAL  SECU- 
RITY AND  INTERNATIONAL  AFFAIRS  DIVISION,  U.S.  GOVERN- 
MENT ACCOUNTING  OFFICE 

Mr.  Rodrigues.  I  would  like  to  have  the  full  statement  entered 
for  the  record  and  proceed  with  a  summary. 

Senator  Pryor.  Without  objection. 

Mr.  Rodrigues.  Thank  you,  Mr.  Chairman.  Mr.  Chairman,  Sen- 
ator Roth,  I  am  pleased  to  be  here  to  discuss  the  current  legislation 
affecting  T&E.  As  you  know,  GAG  has  a  long  history  of  supporting 
independent  testing  as  a  critical  management  control  over  the  ac- 
quisition process.  .  .  , 

Let  me  be  clear.  GAG  generally  supports  most  of  the  acquisition 
reform  proposals.  However,  the  proposed  revisions  to  the  current 
T&E  legislation  should  not  be  enacted  because  they  would  cripple 
the  key  management  control  over  the  system  acquisition  process. 
In  hght  of  the  problems  that  we  continue  to  find,  the  priority  given 
to  test  and  evaluation  should  increase,  not  decrease. 

Our  reviews  have  consistently  shown  that  independent  testing  is 
the  most  reahstic  way  to  determine  whether  DOD  is  getting  what 
it  is  paying  for.  Accordingly,  we  beheve  that  any  change  to  the  cur- 
rent T&E  legislation  should  at  least  preserve,  if  not  strengthen,  the 
fly  before  you  buy  principle. 

Congress  and,  in  particular,  this  Subcommittee  has  been  greatly 
concerned  about  the  performance  of  weapons  systems  being  ac- 
quired by  DOD.  In  an  effort  to  ensure  that  U.S.  mihtary  personnel 
receive  the  best  weapons  systems  possible  and  that  the  Govern- 
ment receives  the  best  value  for  its  defense  dollars,  the  Congress 
has  enacted  legislation  which  defined  operational  test  and  evalua- 
tion and  estabhshed  the  Office  of  the  Director  of  Operational  Test 
and  Evaluation  with  specific  oversight  responsibilities. 

In  response  to  concerns  that  DOD  was  continuing  to  procure 
large   quantities   of  weapons   systems   without   completing   oper- 


i  The  prepared  statement  of  Mr.  Rodrigues  appears  on  page  39. 
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ational  testing,  Congress  passed  additional  legislation  which, 
among  other  things,  required  that  a  major  system  may  not  proceed 
beyond  low-rate  initial  production  until  its  initial  operational  test 
is  completed,  specified  that  operational  test  and  evaluation  of 
major  acquisition  programs  may  not  be  conducted  until  DOT&E  ap- 
proves the  adequacy  of  those  plans,  and  specified  that,  for  covered 
systems  and  major  munitions  or  missiles  programs,  realistic  sur- 
vivability and  lethality  test  and  evaluation,  respectively,  be  con- 
ducted. 

Section  800  of  the  National  Defense  Authorization  Act  of  fiscal 
year  1991  directed  the  Under  Secretary  for  Acquisition  to  appoint 
a  panel  of  experts  to  start  the  process  of  streamlining  defense  ac- 
quisition laws.  This  year,  in  January  1993,  the  panel's  reports  and 
recommendations  addressing  the  numerous  statutes  governing 
DOD  procurements,  including  those  concerning  test  and  evaluation, 
were  issued. 

The  Section  800  Panel  made  several  legislative  proposals  in  the 
T&E  area  that  have  been  included  in  bills  to  reform  the  defense 
acquisition  system.  We  reviewed  these  proposals,  the  Section  800 
report,  and  all  available  supporting  materials.  As  I  will  discuss  in 
detail,  we  have  concluded  that  the  proposed  changes  are  not  justi- 
fied and  appear  to  be  directed  at  perceived  rather  than  documented 
problems.  We  believe  the  proposals  would  dilute  the  effectiveness 
of  test  and  evaluation  as  a  key  management  control,  and  thus 
would  weaken  rather  than  strengthen  the  system  acquisition  proc- 
ess. 

The  panel's  report  stated  that,  in  some  cases,  developers  ex- 
pressed frustration  over  the  delays  and  expense  imposed  on  their 
programs  by  what  they  characterize  as  over-zealous  testers.  We 
would  point  out,  however,  that  testers  can  test  only  what  is  called 
for  in  the  test  and  evaluation  master  plan  which  is  put  together  by 
the  developers,  the  users  and  the  testers.  If  the  developers  promise 
to  develop  a  system  with  specific  capabilities,  it  is  incumbent  upon 
them  to  clearly  demonstrate  those  capabilities  during  the  testing 
program.  To  the  extent  that  they  cannot  do  that,  we  believe  it  is 
unfair  and  inappropriate  for  the  developers  to  consider  the  testers 
over-zealous  simply  because  they  may  be  the  bearer  of  bad  news. 
I  would  now  like  to  address  quickly  the  individual  proposals.  The 
first  proposal  is  to,  "authorize  the  use  of  component,  subsystem  and 
sub-assembly  testing  instead  of  full-up  system  testing  in  determin- 
ing the  survivability  of  high-value  systems."  The  panel  did  not  con- 
sider the  results  of  a  January  1993  national  Research  Council 
study  which  concluded  that  the  existing  live-fire  test  law  was  a  val- 
uable contribution  to  vulnerabihty  assessment.  The  Council  also 
concluded  that  the  law  was  satisfactory  in  its  present  form,  includ- 
ing its  mandate  for  full-up  system  testing,  because  the  waiver  proc- 
ess permitted  exceptions  where  appropriate.  In  Ught  of  the  Council 
study  results  and  our  earlier  work  in  this  area,  we  recommend  that 
Congress  not  change  the  legislation  to  authorize  less  than  full-up 
system  survivability  testing. 

The  second  proposed  change  is  to,  "provide  authority  to  waive 
survivability  testing  after  full-scale  development  begins."  The  panel 
stated  that  this  was  needed  because  the  cost  and  complexity  of  sur- 
vivability testing  may  not  be  known  until  after  some  systems  enter 
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full-scale  development.  We  do  not  see  the  need  for  this  provision 
because  survivability  test  plans  should  be  completed  and  any  basis 
for  a  waiver  from  full-up  system  testing  should  be  evident  prior  to 
the  start  of  full-scale  development. 

We  believe  much  more  attention  needs  to  be  focused  on  identify- 
ing and  addressing  problem  areas  early  in  the  system  acquisition 
process  because  early  fixes  are  less  expensive,  easier  to  implement, 
and  less  disruptive  to  a  program.  In  the  survivability  test  area, 
more  attention  needs  to  be  placed  on  how  to  do  the  testing  in  a 
timely,  efficient  and  effective  manner,  and  less  attention  on  how  to 
obtain  a  waiver  from  that  testing.  We  believe  this  proposal  would 
provide  incentives  to  delay  planning  and  implementing  tests  as 
long  as  possible  and  then  apply  for  a  waiver. 

The  third  proposed  change  is  to,  "provide  for  the  use  of  'different' 
operational  test  and  evaluation  procedures  in  certain  cir- 
cumstances." This  proposal  is  the  one  that  creates  the  most  trouble 
for  us.  We  are  very  concerned  about  the  potential  impact  of  this 
proposal  and  strongly  recommend  that  it  not  be  enacted. 

First,  until  shown  otherwise,  we  believe  that  regular  operational 
test  and  evaluation — that  is,  field  testing  under  realistic  condi- 
tions— ^is  the  best  way  to  ensure  that  an  item  will  be  effective  and 
suitable.  Second,  the  proposal  doesn't  specify  what  is  meant  by  dif- 
ferent operational  test  and  evaluation  procedures.  Lastly,  this  pro- 
posal would  eliminate  the  role  of  DOT&E  in  approving  test  plans 
before  operational  test  begins. 

The  last  proposal  that  I  would  like  to  address  is,  "provide  an  ex- 
ception under  the  LRIP  requirements  for  strategic  defense  missile 
systems."  We  don't  see  any  justification  for  this  proposed.  The  pan- 
el's rationale  is  that  these  systems  have  low  production  quantities. 
That  rationale  is  not  correct  in  that  at  least  some  of  these  pro- 
grams may  involve  production  of  dozens,  if  not  hundreds,  of  units 
and  will  require  rsunp-up  to  full-rate  production.  Naval  ships  and 
satellites  have  been  given  special  legislative  treatment  because 
they  do  not  typically  ramp-up  to  full-rate  production. 

Mr.  Chairman,  in  summary,  we  believe  that  simpler,  more  flexi- 
ble and  more  responsive  procedures  in  the  test  area  should  not 
compromise  the  necessary  discipline  embodied  in  the  current  legis- 
lation. We  have  not  found,  and  the  Section  800  Panel  has  not  pro- 
vided the  analysis  to  show  that  test  and  evaluation  is  one  of  the 
problems  with  the  system  acquisition  process.  On  the  contrary,  our 
ongoing  work  for  this  Subcommittee  on  the  issue  of  low-rate  initial 
production  indicates  that  testing  requirements  may  need  to  be 
strengthened  and  expanded. 

For  many  major  and  non-major  system  acquisition  programs, 
low-rate  initial  production  has  been  permitted  to  begin  and  con- 
tinue based  not  on  a  system's  technical  maturity,  but  on  schedule 
considerations.  Many  of  the  systems  that  enter  low-rate  initial  pro- 
duction prematurely  later  experience  significant  problems  during 
operational  testing.  In  those  cases  where  production  is  well  under- 
way, costly  retrofits  are  required  for  delivered  units.  In  some  cases, 
such  as  the  B-IB  defensive  avionics,  the  problems  identified  could 
not  be  corrected,  resulting  in  deficient  systems  being  fielded. 

To  reduce  the  risk  of  finding  major  operational  problems  after 
production  starts,  we  have  regularly  recommended  as  much  early 
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operational  testing  as  possible.  Nevertheless,  we  continue  to  find 
that  highly  concurrent  acquisition  strategies  are  featured  in  many 
current  major  and  non-major  programs,  with  little,  if  any,  oper- 
ational testing  expected  until  well  after  the  start  of  low-rate  initial 
production. 

In  today's  national  security  environment,  we  believe  there  should 
be  very  few  cases  where  there  is  a  need  to  assume  the  additional 
risks  inherent  in  a  highly  concurrent  acquisition  strategy.  We  be- 
lieve that  a  requirement  to  conduct  at  least  a  phase  of  operational 
testing  before  low-rate  initial  production  starts  for  both  major  and 
non-major  systems  would  be  feasible  in  most  cases  and  would  be 
an  effective  management  control  over  the  premature  start  of  pro- 
duction. 

In  late  1992,  we  reported  on  the  problems  in  acquiring  defense 
systems  and  described  the  entrenched  culture  that  has  so  effec- 
tively resisted  many  attempts  at  reforming  the  system  acquisition 
process.  We  continue  to  see  that  culture  hard  at  work  today  in  this 
attempt  to  significantly  weaken  the  current  T&E  legislation. 

Plain  and  simple,  we  support  increased  accountability  in  the  sys- 
tem acquisition  process.  We  believe  that  if  a  program  is  sold  on 
meeting  a  certain  requirement,  DOD's  commitment  to  that  pro- 
gi-am  should  be  strictly  limited  until  those  requirements  have  been 
adequately  demonstrated.  Based  on  the  interim  results  of  our  ongo- 
ing work,  we  believe  that  DOD's  initial  commitment  to  low-rate  ini- 
tial production  of  a  weapons  system  should  not  be  made  until  de- 
velopmental and  operational  test  results  show  that  the  system  is 
potentially  effective,  will  not  need  costly  or  extensive  changes,  and 
has  a  reasonable  chance  to  successfully  complete  its  remaining  test 
and  evaluation.  We  expect  to  complete  our  ongoing  work  by  the  late 

summer.  j        o       j. 

In  closing,  Mr.  Chairman,  I  would  Uke  to  respond  to  Senator 
Roth's  specific  question  regarding  the  current  organizational  place- 
ment of  the  Office  of  Director  of  Operational  Test  and  Evaluation, 
We  have  found  that  the  organizational  chart  contained  in  the  Sec- 
retary of  Defense's  January  1994  report  to  the  Congress  is  in  error. 
That  is  the  chart  that  appeared  here  earlier  with  the  dotted  line. 

That  chart  is  incorrect.  It  erroneously  shows  DOT&E  reporting 
directly  to  the  Under  Secretary  of  Defense  for  Acquisition  and 
Technology  rather  than  directly  to  the  Secretary.  At  this  time, 
DOT&E  continues  to  report  directly  to  the  Secretary  of  Defense. 

Mr.  Chairman,  that  concludes  my  statement.  I  would  be  happy 
to  answer  any  questions. 

Senator  Pryor.  Mr.  Rodrigues,  say  that  again.  We  were  watching 

Mr.  Rodrigues.  This  chart  that  appeared  in  the  1994  report  to 
the  President  and  Congress  from  the  Secretary  is  incorrect.  We  met 
with  the  Director  of  Administration  and  Management  on  this  issue. 
I  can't  explain  exactly  how  that  chart  got  in  there.  There  were  ap- 
parently a  number  of  charts  that  were  available  for  consideration 
and  use.  This  one  ended  up  there.  It  is  incorrect.  I  beheve  it  is 
against  the  law  if  it  were  enacted. 

Senator  Pryor.  Yes,  sir.  Is  there  any  proposal  in  S.  1587  that 
would  make  the  operational  test  and  evaluation  office  report  di- 
rectly to  the  Secretary  of  Defense? 
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Mr.  RODRIGUES.  Not  that  I  am  aware  of. 

Senator  Pryor.  Maybe  I  am  not  speaking  correctly,  but  I  thought 
last  year's  DOD  Authorization  deleted  DOT&E's  reporting  to  the 
Secretary  of  Defense.  Is  that  correct?  Did  we  do  that? 

Mr.  RODRIGUES.  The  Act  deleted  the  legislative  requirement  that 
DOT&E  report  directly  to  the  Secretary  without  any  intervening 
review  or  approval  in  between  himself  and  the  Secretary.  The  con- 
ferees suggested  that  DOT&E  be  placed  under  the  comptroller's  of- 
fice, but  that  was  just  a  suggestion.  But  it  did  delete  the  legislative 
requirement. 

Senator  Pryor.  I  am  very  confused  about  a  lot  of  that. 

Now,  I  am  going  to  ask  a  series  of  very  short  questions  and  I  am 
going  to  try  to  elicit  from  you,  Mr.  Rodrigues,  some  short  answers. 
Then  I  am  going  to  jdeld  to  my  colleague,  Senator  Roth,  and  then 
we  wiU  turn  to  Mr.  Murray. 

You  think  there  is  a  lack  of  justification  for  the  testing  provisions 
of  S.  1587,  is  that  right? 

Mr.  Rodrigues.  Absolutely,  Senator.  We  went  through  the  Sec- 
tion 800  Panel,  the  information  that  they  had,  and  basically  most 
of  the  information  they  had  was  developed  through  either  written 
input  from  various  parties  throughout  the  Department  or  discus- 
sions with  individuals.  They  talked  to  project  managers,  program 
managers,  program  executive  officers.  They  talked  to  testing  offi- 
cials. 

It  would  be  very  difficult  to  see  where  they  could  have  developed 
a  consensus  for  the  kinds  of  changes  that  they  were  making.  There 
just  wasn't  a  consensus,  not  even  amongst  those  people  that  you 
would  expect  to  be  anti-test  and  evaluation  because  it  affects  their 
programs  and  creates  problems  for  them.  How  they  end  up  saying 
that  these  were  the  kinds  of  things  that  had  to  be  done  just  isn't 
very,  very  clear  at  all  to  us. 

Senator  Pryor.  Do  you  think  that  earlier  operational  testing  pro- 
vides for  better  acquisition  results? 

Mr.  Rodrigues.  Absolutely.  Senator,  once  again  I  think  one  of 
the  problems  we  see,  and  you  addressed  this  very  effectively,  I 
think,  in  the  ASPJ  hearings,  is  that  it  is  very  difficult  to  face  up 
to  the  problems  that  you  identify  during  operational  testing  once 
you  have  got  that  production  line  open  and  running.  It  is  very,  very 
difficult  to  stop  a  program. 

Senator  Pryor.  We  certainly  have  seen  that,  haven't  we? 

Do  you  beUeve  that  operational  testing  is  too  costly? 

Mr.  Rodrigues.  Absolutely  not.  I  think  you  had  a  chart  up  there 
that  showed  the  percentages  for  a  number  of  programs  that  your 
staff  had  developed. 

Senator  Pryor.  Right. 

Mr.  Rodrigues.  Not  a  very  significant  amount  of  money.  I  would 
like  to  point  out  something.  I  was  looking  at  this  chart  and  it  is 
kind  of  interesting.  If  you  look  across  the  programs  that  you  have 
listed  there,  you  look  at  the  B-1,  clearly  not  a  great  success  story. 
The  C-17 — ^you  know,  the  report  card  is  still  out  on  that,  a  pro- 
gram with  tremendous  problems  right  now  under  great  scrutiny 
within  the  Department  as  well  as  here  on  the  Hill.  Exactly  what 
will  happen  with  that — ^you  know,  they  are  still  early  on  in  their 
testing.  The  program's  costs  have  continued  to  skjTocket.  We  con- 
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tinue  to  waive  performance  requirements  on  that  plane.  We  may 
continue  to  waive  more  in  the  future,  so  I  don't  know  where  that 
is  going  to  stand,  but  on  those  two  clearly  a  very  small  percentage 
of  the  total  program  costs. 

On  ASPJ,  we  know  where  that  stood.  You  are  very  familiar  with 
that.  Now,  let  us  look  at  the  Ml.  The  Ml  has  significantly  more 
than  the  others  and  I  would  say  is  not  that  bad  a  program.  It  is 
out  there  and  running.  I  don't  Imow  whether  there  is  a  direct  cor- 
relation. I  have  never  looked  at  it  in  those  terms. 

Senator  Pryor.  I  have  heard  this  and  I  don't  know  if  it  is  true 
or  not,  but  speaking  of  the  B-1,  one  of  the  earlier  B-ls  ran  into 
a  pelican  and  crashed.  Is  that  a  true  story? 

Mr.  RODRIGUES.  There  were  some  problems  early  on  with  it  run- 
ning into  some  kind  of  bird,  yes. 

Senator  Pryor.  Now,  my  question  is  this:  What  sort  of  testing 
did  we  do?  I  have  heard  some  stories  about  this.  How  did  we  try 
to  make  these  B-1  bombers  not  so  vulnerable  to  flying  objects? 

Mr.  RODRIGUES.  I  am  sure  what  you  are  referring  to  is  I  dis- 
cussed with  your  staff— and  I  am  not  sure  it  was  related  to  the 
wing  problem,  but  it  would  have  been  related  to  windshield  prob- 
lems where  they  had  these  so-called  cannons  that  shoot  chickens 
for  certain  kinds  of  tests.  I  have  never  seen  them. 

Senator  Pryor.  We  actually  had  that? 

Mr.  RODRIGUES.  That  was  my  understanding.  I  have  never  seen 
it.  Senator.  I  am  sure  that  is  where  that  comes  from. 

Senator  Pryor.  I  hope  they  are  from  Arkansas  and  Delaware 
anyway,  those  chickens.  [Laughter.] 

Senator  Pryor.  Do  you  think  that  operational  testing  slows  down 
the  acquisition  process? 

Mr.  RODRIGUES.  No,  I  don't.  Clearly,  if  you  run  into  problems, 
you  could  perceive  that  as  slowing  the  process  down.  But  if  the  ob- 
jective is  to  get  a  system  that  works,  and  that  is  what  you  are 
going  to  measure  to,  not  just  getting  systems,  then  I  would  say,  no, 
it  doesn't.  If  it  is  used  to  stop  production  of  bad  systems  and  all 
you  are  interested  in  is  production,  then  you  would  have  to  say, 
yes,  it  does,  but  I  think  what  we  are  after  are  good  systems  that 
work. 

Senator  Pryor.  Now,  Mr.  Rodrigues,  how  long  have  you  been 

with  GAO? 

Mr.  RODRIGUES.  Twenty-four  years. 

Senator  Pryor.  Twenty-four  years,  so  you  have  seen  this  whole 
process  of  the  operational  testing  office  created,  and  30  percent  of 
the  time  it  has  been  without  a  director,  et  cetera.  Now,  what  do 
you  think  the  mind  set  is,  the  cultural  mind  set,  in  the  Department 
of  Defense  about  this  particular  office  and  about  independent  oper- 
ational testing,  your  personal  observation? 

Mr.  Rodrigues.  Clearly,  it  is  perceived  as  a  roadblock  to  getting 
systems  moving  through  the  process,  getting  them  funded,  getting 
them  out  there.  I  don't  know  any  other  way  to  see  that  from  what 
we  have  been  working  on,  the  work  that  we  have  done  on  numer- 
ous systems,  where  we  run  into  programs  that  are  having  test 
problems.  You  know,  you  get  the  accusations  of,  well,  the  testers 
are  ever-zealous,  they  are  not  interpreting  the  results  properly.  We 
went  through  that  on  the  hearings  with  ASPJ  where  there  were 
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those  kinds  of  accusations  flowing  back  and  forth.  Clearly,  they  are 
the  bearers  of  bad  news  and  nobody  likes  bad  news,  and  it  is  unfor- 
tunate that  our  system  works  the  way  it  does. 

I  would  like  to  bring  in  one  thing  here.  I  have  looked  briefly  at 
the  briefing  charts  and  Senator  Roth's  proposal  (S.  1598)  and,  to 
me,  that  is  the  kind  of  acquisition  reform  that  we  need  to  talk  to. 
You  know,  S.  1587  is  nice,  but  what  we  are  talking  about  here  is 
99  percent  of  the  transactions,  but  only  10  percent  of  the  dollars. 
The  other  1  percent  of  the  transactions,  which  is  90  percent  of  our 
dollars,  are  in  the  weapons  acquisitions  that  are  really  not  affected 
by  this  legislation,  other  than  the  proposed  T&E  parts  of  it  which 
we  don't  agree  with. 

But  I  think  that.  Senator  Roth,  your  bill  that  you  have  proposed 
is  a  good  basis  for  beginning  to  debate  the  whole  question  of  acqui- 
sition reform.  What  do  we  really  need  to  do  to  develop  a  system 
that  provides  the  incentives  and  addresses  the  cultural  issues 
which  have  been  so  effective  at  defeating  every  single  effort  or  at- 
tempt to  reform  acquisition  in  the  past?  There  have  been  numerous 
reforms  and  they  have  not  been  successful  and  I  think  a  lot  of  it 
ties  back  to  the  culture,  and  I  think  you  have  to  rethink  the  whole 
thing  about  how  do  you  have  a  system  that  gets  people  to  do  what 
you  want  to  do,  which  is  have  reasonable  estimates  of  cost  and 
schedule  and  performance,  rather  than  sell  a  system  and  then  end 
up  with  problems.  By  the  time  the  problem  is  recognized,  you  are 
so  far  into  the  program  that  you  don't  have  a  choice.  Your  hands 
are  tied  and  you  are  committed  to  it  and  it  is  very  difficult  to  stop 
it. 

Senator  Pryor.  Let  me  jdeld  to  Senator  Roth. 

Senator  ROTH.  Well,  Mr.  Chairman,  I  think  you  have  asked  the 
principal  questions  I  have. 

Mr.  Rodrigues,  thank  you  for  your  testimony.  I  am  bothered 
about  the  culture  because  I  frankly  feel,  until  we  change  the  cul- 
ture, it  doesn't  make  any  difference  what  we  say  or  do.  We  are  not 
going  to  bring  about  the  kinds  of  changes  we  want.  I  take  it  you 
agree? 

Mr.  Rodrigues.  Absolutely. 

Senator  Roth.  One  minor  question.  In  your  testimony,  you  sup- 
ported the  requirement  in  the  fiscal  year  1994  national  defense  au- 
thorization that  appropriate,  rigorous  and  structured  testing  be 
completed  prior  to  low-rate  initial  production  of  electronic  counter- 
measure  systems.  That,  of  course,  would  differ  where  the  current 
practice  occurs  after  production  begins. 

My  question  is  should  the  term  "appropriate,  rigorous  and  struc- 
tured testing"  be  better  defined  by  the  Congress  to  ensure  proper 
compliance? 

Mr.  Rodrigues.  Yes.  Ideally,  if  we  could  do  that,  Senator,  obvi- 
ously the  better  defined  it  is,  the  better  off  we  are.  The  work  that 
we  are  now  doing  looking  at  low-rate  initial  production  for  this 
Subcommittee — in  part  of  that  we  are  looking  at  what  form  that 
operational  testing — if  we  have  a  form  of  operational  testing  prior 
to  any  low-rate,  what  should  that  look  like  and  how  should  it  be 
specified?  So  we  are  dealing  with  that. 

You  know,  I  would  hate  to  jump  out  and  give  an  answer  to  ex- 
actly what  it  should  be  now  because  we  haven't  completed  our 
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work.  I  don't  want  to  do  the  kinds  of  things  I  criticized  the  800 
Panel  for,  collecting  information  and  not  really  having  enough  to 
support  their  position.  I  would  like  to  finish  that,  and  we  are  going 
to  be  addressing  that  issue. 

Senator  ROTH.  Very  good.  Thank  you  for  your  help  not  only  today 
but  in  the  past. 

Mr.  RODRIGUES.  Thank  you.  I  do  have  one  point  that  I  would  like 
to  talk  to.  Mrs.  Preston  talked  about  the  Ml  tank  and  having  to 
destroy  them  in  live-fire  testing. 

Senator  Pryor.  Yes. 

Mr.  RODRIGUES.  I  would  point  out  that  none  of  those  tanks  were 
destroyed. 

Senator  Pryor.  I  didn't  think  they  were. 

Mr.  RODRIGUES.  All  five  of  them  are  in  operation  today.  They  had 
to  be  retrofitted,  but  we  didn't  totally  destroy  them,  and  I  think 
that  is  pretty  important.  I  would  like  to  read  one  other  thing  that 
I  just  happened  to  pick  up  that  I  think  tells  you  something.  This 
includes  the  Ml  and  that  is  why  I  would  like  to  read  it.  It  is  from 
the  Annual  Report  to  the  President  and  the  Congress,  February 
1992,  from  Secretary  Dick  Cheney. 

"The  live-fire  testing  program  continues  to  yield  substantial  ben- 
efits, as  evidenced  in  the  success  in  Operation  Desert  Storm.  The 
Abrams  tank,"  which  is  the  Ml,  "Bradley  fighting  vehicle,  F-15,  F- 
16,  F-18  and  other  weapons  systems  are  among  the  systems  modi- 
fied prior  to  Operation  Desert  Storm  to  improve  their  survivabihty 
in  combat  as  a  direct  result  of  live-fire  testing.  The  long  lead  times 
and  added  complexities  of  weapons  systems  in  development  will 
add  even  more  importance  to  the  need  to  assess  vulnerabiUties  and 
lethalities  early  in  the  development  cycle."  I  think  that  kind  of  says 
it  very  well. 

Senator  Pryor.  Very  good.  We  thank  you,  Mr.  Rodrigues.  You 
may  remain,  if  you  would  like  at  the  table. 

Senator  Pryor.  We  are  going  to  turn  to  our  fiiend  now,  Secretary 
Russell  Murray.  We  know  that  you  have  a  statement,  but  feel  ft-ee 
to  summarize  that  statement  or  to  read  your  statement,  Mr.  Mur- 
ray. We  appreciate  your  being  here,  a  longstanding  ally  in  this  bat- 
tle for  operational  testing. 

TESTIMONY  OF  RUSSELL  MURRAY,i  FORMER  ASSISTANT 
SECRETARY  OF  DEFENSE 

Mr.  Murray.  Thank  you,  Mr.  Chairman.  Perhaps  you  remember 
Groucho  Marx'  hne  when  he  was  a  television  quiz  show  host,  which 
was  "say  the  magic  'woid'  and  a  duck  will  dtop  down  in  front  of 
you."  You  have  said  my  magic  "woid",  and  it  is  operational  testing, 
and  I  am  your  duck,  and  I  am  glad  to  be  here. 

Senator  Pryor.  We  are  glad  that  you  are  here. 

Mr.  Murray.  I  would  like  to  speak  to  you  today  about  what  seem 
to  me,  in  retrospect,  to  be  four  of  the  most  important  policy  issues 
in  operational  testing.  The  first  of  those  is  whether  the  Director  of 
Operational  Test  and  Evaluation  should  be  independent  of  the  Ac- 
quisition Executive  or,  stated  in  broader  terms,  whether  the  oper- 


'The  prepared  statement  of  Mr.  Murray  appears  on  page  45. 
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ational  testers  of  weapons  should  be  independent  of  the  techno- 
logical developers  of  weapons. 

Acquisition  Executives  tend  to  oppose  that  independence.  They 
argue  that,  as  Acquisition  Executives,  they  are,  of  course,  respon- 
sible for  the  creation  of  all  new  weapons.  To  carry  out  that  respon- 
sibility, they  have  to  have  authority  over  all  phases  of  that  cre- 
ation. Because  one  of  those  phases  is  testing,  and  because  testing 
includes  operational  testing,  they  therefore  say  that  they  must 
have  authority  over  the  Director  of  Operational  Test  and  Evalua- 
tion. 

Wrong.  That  manifesto  seriously  prejudices  the  balance  of  influ- 
ence between  the  weapon  makers,  represented  by  the  Acquisition 
Executives,  and  the  weapon  users,  represented  by  the  Director  of 
Operational  Test  and  Evaluation. 

Without  independent  operational  testing  to  weigh  in  on  the  side 
of  the  weapon  users,  the  weapon  makers'  side  of  the  scales  is  too 
heavy  with  money,  and  with  politics,  and  with  constituencies. 
Without  independent  operational  testers,  there  is  nothing  on  the 
other  side  of  the  scales  but  the  weapon  users  out  in  the  field — the 
fighter  pilot,  the  crew  chief,  the  platoon  leader,  the  gunner's  mate. 
In  spite  of  their  numbers,  these  weapon  users,  so  far  from  Wash- 
ington's corridors  of  power,  cannot  by  themselves  match  the  influ- 
ence of  the  weapon  makers. 

The  fighter  pilot  is  no  match  for  the  wealthy  and  influential  mili- 
tary contractors  who  want  to  get  their  production  lines  rolling  as 
soon  and  as  prolifically  as  possible. 

The  crew  chief  is  no  match  for  the  hundreds  of  thousands  of 
those  contractors'  employees  in  dozens  of  states  who  understand- 
ably lean  on  Washington  to  look  out  for  their  economic  livelihood. 

The  platoon  leader  is  no  match  for  the  scientists  and  engineers 
whose  reputations  could  be  in  trouble  if  their  promised  techno- 
logical breakthroughs  should  fail  in  operational  testing. 

And  the  gunner's  mate  is  no  match  for  the  managers  at  all  levels 
whose  job  it  is  to  get  their  new  weapons  deployed  on  schedule. 

In  short,  what  the  weapon  users  out  in  the  field  are  no  match 
for  is  the  weapon  makers'  great  coalition  of  researchers  and  devel- 
opers £uid  manufacturers  who  dwell  within  the  Acquisition  Execu- 
tive's sphere  of  influence.  The  danger  in  that  imbalance  of  influ- 
ence is  that  new  weapon  projects  can  then  grow  into  avalanches, 
sweeping  away  what  little  opposition  there  may  be  in  their  path. 

What  is  needed  is  a  counterbalance,  some  way  of  making  sure 
that  the  weapon  users'  interests  are  not  swept  away  by  that  ava- 
lanche. The  users'  counterbalance  is  independent  operational  test- 
ing. 

But  why  independent?  Instead  of  reporting  to  the  Secretary  of 
Defense,  why  shouldn't  the  Director  of  Operational  Test  and  Eval- 
uation report  to  the  Acquisition  Executive,  who  would  then  balance 
the  competing  interests  of  the  weapon  makers  £md  the  weapon 
users? 

The  fact  that  we  have  yet  to  nominate  a  professional  warrior  to 
be  the  Acquisition  Executive,  and  probably  never  will,  suggests  the 
weakness  in  that  proposition.  The  reason  lies  in  what  we  demand, 
or  what  we  should  demand,  of  any  prospective  Acquisition  Execu- 
tive. To  handle  that  job,  which  is  already  viewed  by  some  people 
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as  virtually  impossible,  we  demand  superb  technological  training 
and  competence,  an  outstanding  record  in  research  and  develop- 
ment, proven  skill  in  industrial  management,  and  experience  in 
managing  defense  acquisition  because  that  is  no  job  for  a  Pentagon 
neophyte. 

But  if,  in  addition  to  all  that,  we  also  say  that  the  Acquisition 
Executive  must  be  a  professional  warrior,  a  person  with  the  back- 
ground and  experience  in  the  field  to  instinctively  understand  what 
a  weapon  user  needs,  then  we  are  almost  certainly  demanding  the 
impossible. 

In  summary  of  this  first  issue  then,  to  ensure  a  proper  balance 
between  the  makers  and  the  users  of  weapons,  the  Acquisition  Ex- 
ecutive and  the  Director  of  Operational  Testing  must  be  independ- 
ent. The  qualifications  we  should  demand  of  these  two  positions 
are,  in  effect,  mutually  exclusive.  The  Director  of  Operational  Test- 
ing should  be  a  creature  of  the  community  that  uses  rather  than 
creates  weapons.  And,  as  recommended  24  years  ago  by  the 
Fitzhugh  Blue  Ribbon  Panel,  the  Director  of  Operational  Test  and 
Evaluation  should  report  directly  to  the  Secretary  of  Defense. 

The  second  pohcy  question  is  when  in  the  development  process 
should  operational  testing  occur.  I  sense  that  operational  testing  is 
increasingly  being  viewed  only  as  little  more  than  a  "final  exam". 
Though  that  is  an  important  milestone,  operational  testing  can  be 
even  more  important  at  the  very  inception  of  a  project.  Charles 
Myers'  testimony  earlier  before  this  Committee  illustrates  the 
point.  He  said,  "The  question  of  [visual]  acquisition  of  tanks 
seemed  important  relative  to  being  able  to  use  a  Maverick  anti- 
tank missile.  So  we  conducted  a  modest  conceptual  test  at  my  farm. 
We  parked  a  dirty  old  green  pickup  truck  in  the  tall 
grass  .  .  .  and  we  went  out  to  a  local  airport  and  leased  a  Cessna 
152.  My  wife  and  I  went  out,  recorded  the  data,  and  correlated  it 
with  a  large  photograph  of  our  farm  area  where  the  terrain  is 
much  like  Europe."  Chuck  went  on  to  say,  "We  came  up  with  a  set 
of  numbers  that  would  indicate  that  you  could  probably  see  a  tank 
ft-om  about  7,000  feet  slant  range.  That  is  less  than  the  distance 
[needed]  to  use  the  Maverick.  Many  years  later  .  .  .  those  num- 
bers were  verified  by  Air  Force  testing.  The  problem  was  that  the 
people  involved  in  the  early  development  testing  .  .  .  didn't  really 
address  the  operational  problem  of  a  pilot  who  has  to  fire  first  on 
a  tank  before  the  air  defense  can  shoot  at  him."  That  is  the  end 
of  Charles  Myers'  quote. 

Now,  I  don't  know  what  that  operational  test  cost — perhaps  a  few 
hundred  dollars.  Yet,  had  the  Air  Force  paid  attention  to  his  re- 
sults, they  could  have  saved  the  hundreds  of  millions  of  dollars 
that  they  wasted  on  a  hopeless  system  and  instead  gotten  busy  on 
an  alternative  that  a  pilot  could  actually  use. 

Section  138  of  Title  10  of  the  U.S.  Code  defines  operational  test- 
ing as  ".  .  .  the  field  test,  under  realistic  combat  conditions,  of  any 
item  of  (or  key  component  of)  weapons,  equipment,  or 
munitions  ..." 

As  Special  Counsellor  of  the  House  Armed  Services  Committee 
when  that  section  was  drafted,  I  can  testify  that  the  words  "(or  key 
component  of)"  were  inserted  specifically  to  encourage  early  concep- 
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tual  operational  testing  rather  than  just  final  exams  for  fully  com- 
pleted weapons. 

The  third  policy  issue  is  the  charge  that  operational  testing  adds 
to  the  already  excessive  length  of  weapon  development  and  farther 
delays  fielding  new  weapons. 

As  long  as  the  criterion  is  merely  "fielding  new  weapons,"  the  al- 
legation of  delay  could  well  be  true.  But  that  is  the  wrong  criterion. 
The  right  criterion,  just  a  little  bit  longer,  is  "fielding  new  weapons 
ready  for  combat."  If  operational  testing  shows  that  the  weapon 
isn't  ready  for  combat,  it  should  be  delayed. 

In  fact,  from  delaying  the  fielding  of  combat-ready  weapons, 
operational  testing  accelerates  the  process  because  it  takes  far  less 
time  to  correct  faults  discovered  early  in  operational  testing  before 
production  and  deployment  starts  than  it  takes  to  retrofit  faulty 
weapons  out  in  the  field.  After  all,  if  you  are  just  out  to  deploy 
something  as  fast  as  possible,  you  could  always  cut  out  the  develop- 
ment testing,  too. 

The  fourth  policy  issue  is  the  charge  that  operational  testing 
costs  too  much,  particularly  in  this  era  of  declining  budgets.  But 
operational  testing  accounts  for  only  a  tiny  fraction  of  the  total  cost 
of  a  weapons  program,  as  you  have  shown.  And  every  time  that  it 
discloses  a  fault  in  a  new  weapon  that  can  be  corrected  before  (in- 
stead of  after)  the  production  line  begins  to  roll,  the  saving  swamps 
the  relatively  trivial  cost  of  the  testing. 

Mr.  Chairman,  I  can't  conclude  without  mentioning  the  pending 
proposal  in  S.  1587  to  allow  waivers  from  current  operational  test- 
ing requirements.  The  draft  that  I  have  seen  would  return  the  bal- 
ance between  the  makers  and  the  users  of  weapons  to  the  lopsided 
inequality  of  a  decade  ago  before  this  Committee  led  the  drive  to 
level  that  balance  by  legislating  independent  operational  testing. 

The  sweep  of  this  new  proposal  is  breathtaking.  There  is  no  ap- 
parent hmit  to  the  possible  degradation  or  even  abandonment  of 
operational  testing.  The  only  description  of  the  revised  procedures 
is  that  they  could  be,  quote,  "different,"  unquote.  Nor  is  there  any 
limit  to  the  importance  of  the  programs  subject  to  such  procedural 
vandalism.  These  new  provisions  could  apply  to  any  program. 

The  draft  stipulates  four  justifications  for  such  waivers.  The  first 
would  be  if  normal  operational  testing  is  alleged  to  be  "unreason- 
ably expensive."  But,  as  I  have  noted,  it  can  cost  a  lot  more  to  fix 
a  faulty  weapon  in  the  field  than  it  could  possibly  cost  to  test  it 
properly  before  it  goes  into  production. 

The  second  justification  would  be  if  normal  operational  testing  is 
alleged  to  be  "impracticable."  But  the  lack  of  any  definition  of  the 
standards  for  impracticability  is  just  an  open  invitation  to  ham- 
string operational  testing. 

The  third  justification  would  be  in  cases  where  normal  oper- 
ational testing  would  "cause  unwarrsuited  delay."  But  the  question 
is  delay  in  what?  Delay  in  fielding  a  weapon,  or  delay  in  fielding 
a  combat-ready  weapon? 

The  fourth  justification  is  just  so  sweeping  that  I  can  only  won- 
der why  the  drafters  even  bothered  with  the  other  three.  This  one 
would  allow  a  waiver  if  normal  operational  testing  would,  "be  un- 
necessary because  of  the  acquisition  strategy  for  that  system."  As 
I  read  that  language,  it  would  let  the  Defense  Department  gut 
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operational  testing  any  time  it  wanted  to  by  simply  writing  it  out 
of  their  "acquisition  strategy." 

The  draft  does  conclude  with  one  restriction  on  such  waivers. 
They  would  not  apply  to  programs  "beyond  low-rate  initial  produc- 
tion". The  clear  implication  is  that  operational  testing  would  apply 
only  to  the  "final  exam",  which  this  language  would  now  defer  to 
the  end  of  low-rate  initial  production. 

As  I  noted  earlier,  operational  testing  should  play  its  role  from 
the  very  beginning  of  a  program  and  not  just  as  some  "final  exam". 
If  the  operational  tester's  hands  are  to  be  tied  until  a  weapon  com- 
pletes low-rate  initial  production,  his  only  choice  will  be  to  run  for 
cover  because,  by  that  time,  the  avalanche  will  be  in  full  force. 

You  call  it  the  Mack  truck  loophole.  Mr.  Chairman,  I  suggest 
that  it  is  a  loophole  big  enough  to  drive  a  DIVAD  through.  [Laugh- 
ter.] 

Mr.  Murray.  In  summary,  Mr.  Chairman,  then,  operational  test- 
ing must  be  independent.  It  should  be  used  from  the  beginning  of 
weapon  development.  It  doesn't  delay  deployment  of  weapons  that 
work;  it  accelerates  it.  It  doesn't  waste  money;  it  saves  it.  And  I 
urge  you,  beware  of  proposals  that  would  cripple  the  user's  influ- 
ence. 

Thank  you,  sir. 

Senator  Pryor.  Thank  you  very,  very  much  for  your  statement. 
You  have  spent  a  lot  of  years  studying  these  issues,  Mr.  Murray. 
We  have  already  heard  that  the  testing  office  opposes  this  change. 
We  have  heard  that  the  Acting  Inspector  General  has  opposed  this 
change.  We  have  heard  that  the  General  Accounting  Office  has  op- 
posed this  change — the  DIVAD  amendment,  we  might  call  it  now, 
or  the  Mack  truck  waiver,  whichever. 

Now,  how  does  language  like  this  get  to  become  a  part  of  a  major 
defense  acquisition  reform  bill?  What  happens?  Tell  us  about  the 
bureaucracy  and  the  politics  of  the  procurement  process  that  gets 
language,  over  the  objection  of  those  offices,  into  a  proposed  de- 
fense acquisition  program.  How  did  this  happen? 

Mr.  Murray.  I  don't  know  how  it  happened  in  this  particular 
case.  I  can  only  guess.  But,  of  course,  you  are  all  aware  of  the  in- 
tense opposition  to  operational  testing.  We  have  seen  this  for,  what 
is  it,  24  years  now  since  the  1970  Blue  Ribbon  Panel? 

As  I  mentioned,  there  is  a  lot  of  money  and  there  is  a  lot  of  poli- 
tics and  there  are  a  lot  of  constituencies  that  want  to  hamstring 
operational  testing.  They  presumably  have  gotten  to  whoever  the 
drafter  of  that  language  was  and  had  their  way  with  it.  If  you  sit 
still  for  that,  it  is  going  to  be  the  end  of  operational  testing. 
Senator  Pryor.  I  appreciate  that. 

Mr.  Murray.  I  wish  I  could  give  you  a  concrete  answer.  I  mean, 
I  don't  know  who  wrote  it.  I  would  hke  to  know. 

Senator  Pryor.  Well,  it  is  a  mystery  that  language  that  was  op- 
posed by  so  many  individuals  is  here  in  the  legislation,  in  Section 
3013. 

Mr.  Murray.  Well,  one  thing  that  strikes  me  about  it  is  it  seems 
extraordinarily  amateurish.  If  I  were  trying  to  kill  OT&E,  I  think 
I  would  be  a  Httle  more  subtle  about  it  than  that.  That  is  really 
childish. 
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Senator  Pryor.  Let  me  ask  a  couple  of  a  specific  questions.  Why 
do  you  feel  that  the  concept  of  operational  testing  is  so  outmatched 
by  the  acquisition  community?  How  did  this  come  about?  Is  the  ac- 
quisition community  the  contractors? 

Mr.  Murray.  Yes,  indeed. 

Senator  Pryor.  Are  there  people  within  the  Department  of  De- 
fense who  say  we  have  got  to  do  more  in  acquisition?  Do  we  have 
a  schedule-driven  system  now  that  we  have  to  compete  with? 

Mr.  Murray.  Well,  lots  of  weapons  managers  are  graded,  if  you 
will,  on  the  success  they  have  in  meeting  their  assigned  schedules 
in  developing  new  weapons.  So,  obviously,  if  the  weapon  develop- 
ment is  late,  that  is  a  black  mark  against  them.  So,  that  is  one 
thing  that  leads  toward  a  reluctance  to  engage  in  operational  test- 
ing. I  mean,  it  might  come  up  with  a  wrong  answer. 

As  I  mentioned,  there  is  a  lot  of  money.  There  are  a  lot  of  produc- 
ers who  don't  want  to  see  their  products  delayed  by  unfavorable 
operational  testing.  This  feeling  is  deep-seated.  I  can  recall  at  the 
time  that  operational  testing  was  assigned  to  my  Office  of  Program 
Analysis  and  Evaluation,  there  was  a  report  from  the  Defense 
Science  Board  written  by  Mr.  Delauer,  who  later  became  the  Acqui- 
sition Executive,  that,  in  effect — I  can't  recall  the  exact  language, 
but,  in  effect — accused  our  office  of  wanting  this  responsibility  as 
a  way  of  canceling  weapons  that  we  could  not  otherwise  cancel  in 
order  to  cut  the  budget.  I  deeply  resented  that,  but  nonetheless  it 
was  an  indication  of  the  kind  of  feeling  that  there  is  out  there,  and 
it  is  not  going  to  go  away. 

Senator  Pryor.  What  about  the  request  this  morning  by  Mrs. 
Preston  when  she  stated  the  policy  of  the  Department  of  Defense 
that  we  need  more  contractor  involvement  or  support  in  oper- 
ational testing? 

Mr.  Murray.  It  is  interesting.  She  also  mentioned  something 
which  I  think  is  closely  aligned  with  that,  and  that  is  the  idea  of 
combining  developmental  and  operational  testing  in  order  to  save 
money.  That  indicates  a  fundamental  misunderstanding  of  the  dif- 
ference between  operational  and  development  testing. 

Development  testing  is  done  under  carefully  controlled  conditions 
and  its  basic  purpose  is  to  determine  whether  or  not  the  new  weap- 
on meets  its  contractual  specifications.  So  you  have  got  scientists 
and  engineers  and  all  sorts  of  precision-measuring  devices  to  find 
out  whether  the  thing  flies  far  enough  or  is  sufficiently  accurate. 
It  is  done  under  the  most  carefully  controlled  conditions.  It  has 
nothing  to  do  with  wartime  conditions. 

Operational  testing,  on  the  other  hand,  has  nothing  to  do  with 
whether  a  weapon  meets  its  contract  specifications.  It  has  to  do 
with  whether  the  thing  is  usable  in  the  field  and  whether  there  is 
some  easy  way  for  the  enemy  to  defeat  it.  It  should  not  be  run  with 
contractor  personnel  unless  contractor  personnel  are  going  to  par- 
ticipate in  the  field  in  that  war.  Now,  if  you  want  to  specify  that, 
fine,  then  you  use  contractor  personnel. 

She  said,  I  noticed,  contractor  personnel  are  necessary  for  the  lo- 
gistic support.  Logistic  support  is  a  broad  term,  but  whether  or  not 
a  machine  is  logistically  supportable  in  the  field  is  definitely  some- 
thing to  be  determined  in  the  course  of  operational  testing.  So  I 
just  disagree  with  that  entirely. 
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Senator  Pryor.  Do  you  think  if  the  troops  out  in  the  field  could 
have  a  vote,  they  might  strike  this  language? 

Mr.  Murray.  I  think  you  might  see  a  120-millimeter  shell  go 
right  through  the  middle  of  it. 

Senator  Pryor.  Mr.  Murray,  we  want  to  thank  you.  You  have 
been  a  valuable  resource  for  us  and  certainly  a  valuable  addition 
to  this  morning's  hearing.  We  thank  you  so  much  for  your  involve- 
ment. 

Senator  Pryor.  Mr.  Rodrigues,  we  thank  you,  and  those  from  the 
Department  of  Defense  who  have  been  here  this  morning.  We  hope 
we  will  work  this  matter  out  with  the  ultimate  result  of  better 
operational  testing.  I  am  sure  I  am  speaking  for  Senator  Roth,  we 
want  to  express  our  gratitude  to  all  our  witnesses  this  morning. 

Thank  you  very  much. 

[Whereupon,  at  12:27  p.m.,  the  Subcommittee  was  adjourned.] 
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PREPARED  STATEMENT  OF  MRS.  COLLEEN  A.  PRESTON 

Chairman  Pryor,  Senator  Stevens,  Members  of  the  Committee,  I  am  happy  to  be 
here  today  to  present  the  Department  of  Defense  position  on  operationsd  test  and 
evaluation. 

Mr.  Chairman,  with  your  permission  I  would  like  to  ask  that  my  complete  state- 
ment be  included  for  the  record.  My  statement  addresses  each  of  the  issues  you 
have  raised  with  Secretary  Perry  and  Deputy  Secretary  Deutch,  including,  your  con- 
cerns with  respect  to  the  Airborne  Self-Protection  Jfimmer  (ASPJ),  the  appointment 
and  organizational  structure  of  the  Office  of  Operational  Test  and  Evaluation,  the 
use  of  simulation  in  the  development  and  testing  of  weapons,  and  the  various  initia- 
tives to  reform  the  testing  legi^ation. 

I  will  confine  my  verbal  comments  to  a  quick  overview  of  each  of  the  issues  ad- 
dressed in  my  statement,  and  then  focus  on  the  Department's  views  on  acquisition 
reform  as  it  relates  to  testing. 

ASPJ 

Mr.  Chairman,  as  Deputy  Secretary  Deutch  wrote  to  you  on  March  15,  1994,  the 
ASPJ  program  has  had  a  controversial  history.  Because  the  system  failed  to  satisfy 
its  operational  test  criteria,  the  Department  is  not  procuring  the  system.  The  deci- 
sions to  terminate  the  program,  but  also  to  buy  a  limited  number  of  the  systems, 
95  in  total,  were  made  before  the  Clinton  Administration  took  office. 

I  can  assure  you  that  there  are  no  plans  within  the  Department  of  Defense  to  ac- 
quire additional  ASPJs.  As  you  know,  the  Navy's  F-14D  requires  a  self-protection 
jammer,  and  is  configured  to  accept  only  the  ASPJ  system.  As  part  of  the  F-14D 
operational  evaluation,  it  will  be  tested  with  the  ASPJ.  If  it  is  determined  that  the 
F-14D  is  more  survivable  with  the  ASPJ  than  without  it,  it  will  be  installed  for  fleet 
use.  The  Deputy  Secretary,  however,  has  committed  to  instruct  the  Navy  to  with- 
hold any  F-14D  testing  of  this  system  until  a  Director  of  Operational  Test  and  Eval- 
uation has  been  confirmed,  is  on  the  job,  and  offers  recommendations  as  to  the  ade- 
quacy and  utility  of  such  tests. 

In  addition,  the  F/A-18  has  an  urgent  requirement  for  self-protection.  This  re- 
quirement will  be  satisfied  by  developing  a  new  Test  and  Evaluation  Master  Plan 
(TEMP),  which  we  expect  to  take  approximately  18  months.  The  TEMP  will  provide 
for  a  comparative  analysis  that  will  allow  an  informed  analysis  of  all  electronic  war- 
fare alternatives  for  use  on  current  and  future  Navy  aircraft.  Any  decision  on  an 
electronic  warfare  system  will  be  based  on  a  new  Cost  and  Operational  Effectiveness 
Analysis. 

Finally,  the  Department's  position  remains  that  Foreign  Military  Sales  (FMS) 
funds  will  not  be  used  to  support  sales  of  ASPJ  to  countries  because  the  Department 
does  not  want  to  imply  a  warranty  on  a  system  that  will  not  be  prooired  for  U.S. 
aircraft.  The  Department  remains  committed  to  the  policy  that  no  foreign  military 
sales  or  commitments  for  foreign  military  sales  of  defense  systems  will  be  made 
prior  to  the  successful  completion  of  operational  test  and  evaluation,  and  the  specific 
approval  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology).  Consider- 
ation of  the  use  of  FMS  fUnds  to  support  the  integration  of  a  commercially  piir- 
chased  system  into  an  American  FMS  aircraft  will  be  made  on  a  case-by-case  basis. 

OPERATIONAL  TEST  AND  EVALUATION  ORGANIZATION 

The  Department  of  Defense  is  strongly  committed  to  independence  of  the  Director, 
Operational  Test  and  Evaluation.  The  President's  choice  to  be  the  new  Director, 
Operational  Test  and  Evaluation,  is  at  this  time  going  through  the  Administration's 
review  and  clearance  process.  In  addition,  earUer  this  month  Mr.  Deutch  signed  a 
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memorandum  specifying  that  the  Director,  while  working  closely  with  the  Under 
Secretary  of  Defense  for  Acquisition  and  Technology,  would  report  directly  to  the 
Secretary  and  Deputy  Secretary  of  Defense. 

SIMULATION  IN  THE  DEVELOPMENT  AND  TESTING  OF  ACQUISITION  SYSTEMS 

Simulation  is  a  technology  the  Department  will  increasingly  be  using  to  reduce 
both  the  cost  and  time  of  development  as  well  as  of  test  and  evaluation.  Simulation 
will  be  used  to  support,  but  not  to  replace,  realistic  operational  testing. 

Much  of  the  cost  and  time  of  both  developmental  and  operational  test  is  the  result 
of  retesting  equipment.  Initial  tests  often  identify  failure  modes  (equipment  faults 
or  employment  deficiencies)  requiring  the  redesign  of  the  equipment  or  the  retrain- 
ing of  the  users  before  retesting  can  verify  that  performance  requirements  are  met. 
This  test-fix-test  process  increases  the  cost  and  the  length  of  the  acquisition  process. 
Simulation  offers  the  potential  of  a  cheaper  and  quicker  way  to  find  failure  modes 
than  does  field  testing.  It  is  cheaper  because  our  simulation  technology  now  allows 
experiments  (called  aistributive  interactive  simulation)  without  having  to  move 
large  numbers  of  personnel  and  equipment  around  the  country  to  a  test  range.  It 
is  quicker  because  simulation  can  begin  before  hardware  is  produced. 

Simulation  also  helps  us  to  find  and  focus  on  problems  early  in  development. 
Evaluation  during  development  currently  suffers  because  the  operational  con- 
sequences of  what  is  found  in  developmental  test  are  largely  unknown.  Simulation 
can  suggest  the  operational  significance  of  developmental  tests.  Further,  our  biggest 
problem  is  our  tendency  to  bring  immature  systems  to  operational  test.  Operational 
tests  find  problems,  but  in  most  cases,  the  find-and-fix  process  deUvers  a  system 
that  will  not  be  operationally  suitable  or  operationally  effective  imtil  the  second  or 
third  try.  Simulation  can  be  used  to  avoid  costly  test  milures  that  occur  after  design 
freeze — requiring  expensive  redesign  and  tooling  changes. 

REFORM  OF  THE  OPERATIONAL  TEST  AND  EVALUATION 

As  I  indicated  previously,  the  Secretary  and  Deputy  Secretary  of  Defense  believe 
that  independent  operational  test  and  evaluation  is  a  critical  part  of  the  acquisition 
process.  1  share  that  belief.  As  the  cost  of  new  weapons  systems  increases  and  the 
Department  of  Defense  budget  decreases,  DOD  cannot  afford  to  field  weapons  that 
are  not  operationally  suitable  and  effective.  The  current  statutory  regime  for  oper- 
ational and  live  fire  testing,  however,  has  not  evolved  with  the  changes  in  our  acqui- 
sition process  that  have  been  made,  and  are  being  recommended,  to  meet  our  new 
national  security  challenges — poUtical,  economic,  and  military.  The  result  is  that 
DOD  is  spending  far  more  in  both  dollars  and  time  than  is  necessary  to  achieve  the 
goal  we  all  agree  upon. 

The  DOD  Advisory  Panel  on  StreamUning  and  Codifying  Acquisition  Laws  (the  so- 
called  Section  800  Panel)  examined  the  three  current  general  testing  statutes  (10 
U.S.C.  2366,  2399,  and  2400).  The  panel  found  that  the  statutes  promote  imneces- 
sary  oversight,  have  excessive  definitional  and  reporting  detail,  do  not  allow  the  De- 
partment to  exercise  sound  managerial  discretion  in  test  appUcation  and  implemen- 
tation, and  limit  participation  of  system  contractors  in  operational  test  where  such 
involvement  is  necessary  and  prudent. 

The  Panel  recommended  acfoption  of  a  streamlined  testing  statute  that  set  forth 
the  basic  rule  that  both  vulnerability/lethality  and  operational  testing  must  occur 
before  proceeding  beyond  low-rate  initial  production.  The  proposed  statute  then 
vests  discretion  in  the  Secretary  of  Defense  to  implement  the  required  testing. 
Broad  guidelines  in  specific  areas — such  as  contractor  involvement  and  authority  to 
modify  test  requirements — are  provided.  These  guidelines  state  general  principles, 
but  specific  implementation  is  left  to  the  Secretary  of  Defense.  Alternatively,  the 
Panel  recommended  that  each  of  the  three  major  testing  statutes  be  modified  slight- 
ly to  provide  some,  but  not  all,  of  the  flexibility  allowed  in  the  streamlined  statute. 

When  considered  by  the  Department,  the  Acquisition  Reform  Senior  Steering 
Group,  which  includes  representatives  of  the  services  and  all  OSD  organizations  in- 
volved in  the  acquisition  process,  voted  in  fav&r  of  the  single  streamlined  statute, 
with  the  exception  of  the  Inspector  General  and  the  Director,  Operational  Test  and 
Evaluation.  Mr.  Deutch  accepted  the  recommendation  of  the  Steering  Group.  The 
DOD  position  was  subsequently  adopted  as  the  Administration  position,  and  has 
been  presented  as  such  to  Congress  in  the  DOD  comments  on  S.  1587. 

All  of  the  acquisition  reform  legislative  proposals  being  considered  by  Congress 
have  made  some  changes  to  the  testing  statutes.  None  of  the  legislative  proposals 
adopt  either  the  streamlined  statute  proposed  by  the  Section  800  Panel  nor  the  al- 
ternative. S.1587  comes  closest,  adopting  the  Section  800  alternative  for  Uve  fire 
testing  (10  U.S.C.  §2366)  and  low-rate  initial  production  (10  U.S.C.  §2400),  but  the 
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S.1587  changes  to  operational  test  and  evaluation  (10  U.S.C.  §2399)  provide  very 
little  flexibility  and  no  contractor  support  relief. 

I  believe  that  there  are  four  problems  with  the  current  statutes  that  must  be 
solved. 

Problems  with  Current  Testing  Statutes 

First,  the  current  laws  are  inflexible  and  unrealistic,  causing  DOD  to  expend  a 
lot  of  money  for  little  gain. 

The  MlAl  live  fire  test  program  required  the  firing  of  48  shots  against  five  com- 
bat loaded  Abrams  tanks.  The  approximate  cost  of  the  test  program  was  $27  million 
($12  million  for  the  test  program  and  $15  million  for  the  tanks).  A  more  systematic 
approach  to  assessing  venicle/crew  vulnerability  with  fewer  shots  into  ftill-up  com 
bat  vehicles  would  have  accomplished  the  same  objective  with  a  significant  reduc- 
tion in  cost.  For  example,  using  components,  subsystems,  ballistic  hulls  and  turrets, 
and  one  or  two  full-up  tanks  would  have  provided  the  same  information. 

The  requirements  for  live  fire  and  operational  test  contained  in  the  current  stat- 
utes drive  the  test  community  to  expend  far  more  resources  than  are  necessary  to 
achieve  the  goal  of  identifying  risks  m  the  system.  Test  is  not  an  end  in  itself.  The 
purpose  of  any  test  is  to  allow  the  decision  maker  to  make  an  informed  judgment 
concerning  viimerabilityAethality  and  operationail  suitability  and  effectiveness  of  the 
system  being  evaluated. 

Second,  the  current  laws  artificially  require  segregation  of  developmental  and 
operational  testing. 

On  the  F-22  total  7000  hour  test  program,  only  540  hours  are  allotted  to  oper- 
ational testing.  The  operational  suitaoility  and  effectiveness  will  be  determined  on 
the  basis  of  540  hours  of  maintenance  data  rather  than  on  7000  hours  of  data. 

Testing  is  a  continuous  process.  Knowledge  gained  firom  one  test  is  a  building 
block  for  evaluating  the  knowledge  gained  fi-om  follow-on  tests.  Artificially  segregat- 
ing test  data  reduces  the  validity  and  usefulness  of  all  the  tests.  Making  judgments 
on  only  a  part  of  the  information  gained  through  testing  makes  the  decision  less 
informed  than  it  could  be.  Also,  artificial  separation  of  developmental  and  oper- 
ational testing  encourages  the  operational  test  to  be  seen  as  a  final  exam  rather 
than  as  an  opportunity  to  gain  insight.  This  perception  causes  barriers  between  the 
testers  and  tne  developers  that  inhibits  the  timely  correction  of  problems  discovered 
in  testing. 

Third,  the  current  laws  invade  on  the  Service's  prerogatives  for  systems  that  are 
not  being  overseen  by  OSD  by  requiring  approval  of  testing  plans  by  OSD  testing 
organizations  as  opposed  to  the  Service  testin^j  organizations. 

DOD  has  established  a  structure  for  oversight  of  programs  that  makes  the  deci- 
sion level  for  approval  of  programs  in  the  Services  and  OSD  commensurate  with  the 
doUair  value  and  significance  of  the  system  being  acquired.  This  structure  allows  the 
decision  makers  at  the  highest  levels  to  concentrate  on  those  systems  that  appro- 
priately require  their  attention.  For  example,  the  Under  Secretary  of  Defense  for  Ac- 
quisition &  Technology  is  the  milestone  decision  authority  for  only  those  programs 
Uiat  meet  the  definition  of  ACAT  ID  programs  (about  40-45  programs,  worth  ap- 
proximately $400  billion).  The  current  testing  live-fire  and  operational  testing  stat- 
utes require  approvals  from  OSD  testing  organizations  on  programs  that  normally 
would  not  come  to  OSD  for  review — rather,  the  Services  would  be  the  designated 
decision-maker  on  systems  below  the  ACAT  ID  level. 

We  believe  that  in  the  interest  of  streamUning  and  assuring  accountability  for  de- 
cision-making with  respect  to  a  program,  that  decisions  regarding  live-fire  and  oper- 
ational testing  should  be  at  the  same  organizational  level  that  other  critical  deci- 
sions regarding  the  program  are  being  made.  Testers  are,  and  should  continue  to 
be  independent  of  those  doing  system  development.  There  is  no  reason  to  believe, 
however,  that  the  testing  community  within  the  services  is  any  less  able  to  protect 
the  interests  of  the  testing  conmiunity,  or  any  less  independent  that  the  testing  or- 
ganizations at  the  OSD  level. 

Fourth,  the  current  laws  do  not  allow  the  Services  to  use  appropriate  contractor 
support  in  two  different  contexts.  First,  the  use  of  services  contractors  for  both  de- 
velopmental and  operational  testing.  Second,  the  use  of  a  contractor  who  is  produc- 
ing or  developing  the  weapon  system. 

On  the  Advanced  Medium  Range  Air-to-Air  Missile  (AMRAAM),  the  contractor 
who  was  doing  data  analysis  for  the  Navy  during  developmental  testing  was  a  sub- 
contractor for  the  contractor  hired  to  do  operational  testing  data  analysis.  The  cur- 
rent statute  on  contractor  involvement  in  data  analysis  required  the  Navy  to  stop 
operational  testing.  This  situation  was  not  resolved  for  3-4  months  until  the  person 
doing  the  data  analysis  resigned  from  his  company  and  established  his  own  firm 
that  was  then  given  the  data  analysis  contract. 
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One  of  the  F-22  contractors  has  a  full  mission  simulator  (FMS)  that  is  essential 
to  evaluate  the  F-22  against  advanced  threats,  and  in  a  dense  environment  that 
cannot  be  tested  in  the  field.  This  FMS  is  the  only  existing  simulator  that  can  sat- 
isfy the  F-22  requirement.  The  FMS  is  located  in  the  system  contractor's  faciUty 
and  operated  by  system  contractor  personnel.  The  cost  of  using  the  FMS  during 
operational  testing  in  the  contractor  facility  with  contractor  personnel  is  $6.5  mil- 
lion. In  order  for  the  Air  Force  to  do  operational  test,  however,  the  Air  Force  must 
replace  contractor  personnel  operating  the  FMS — at  a  training  cost  of  $10  million, 
and  build  an  FMS  at  a  Government  facility  at  a  cost  of  $24  million  (if  a  current 
simulator  is  upgraded)  to  $68  mUlion  (for  a  brand  new  facility).  The  only  other  op- 
tion is  to  use  digital  simulations  that  will  result  in  loss  of  a  credible  man-in-the- 
loop  capability  and  the  potential  fidelity  of  the  data. 

The  experience  level  of  user  personnel  is  often  Umited  or  non-existent  at  oper- 
ational test.  Judicious  use  of  contractor  personnel  to  broaden  the  experience  base 
during  the  test,  as  long  as  the  integrity  of  the  test  is  maintained,  provides  a  more 
realistic  assessment  of  the  operability  of  the  system.  Use  of  contractor  personnel 
fi-om  a  contractor  that  is  not  involved  in  either  the  development  (excluding  testing) 
or  production  of  the  system,  to  support  both  developmental  and  operational  test, 
adds  to  the  value  of  test  evaluation  because  the  evaluators  are  more  knowledgeable 
about  the  system  being  evaluated. 

In  order  to  solve  these  four  problems,  and  otherwise  clarify  and  strengthen  the 
testing  statutes,  I  am  proposing  that  DOD  support  changes  to  the  current  statutes 
that  would  do  the  following: 

LIVE  FIRE  TESTING 

•  Substitute  the  word  "vuhierability"  for  "survivabihty"  (in  10  U.S.C.  2366)  each 
time  it  appears,  to  clarify  that  live  fire  testing  is  intended  to  address  specifically 
the  probaoility  of  a  kill  given  a  hit,  including  both  platform  and  crew  vulnerability, 
rather  than  the  entire  spectrvmi  of  considerations  usually  encompassed  in  the  term 
"survivabihty"  (target  acquisition,  maneuver,  jamming,  and  tactics). 

•  Require  that:  "Testing  be  accompUshed  first  at  the  component,  subsystem,  and 
subassembly  level,  cvdminating  with  tests  of  the  complete  system  configured  for  com- 
bat" (amend  10  U.S.C.  2366  by  adding  a  new  paragraph  3). 

•  Authorize  the  Secretary  of  Defense  (delegable  to  the  appropriate  milestone  deci- 
sion authority)  to  "waive  viilnerabihty  or  lethality  tests  on  a  complete  system  config- 
ured for  combat  if  the  Secretary  determines  that  such  testing  would  be  unreason- 
ably expensive  and  impractical,  and  testing  of  components,  subsystems,  and  assem- 
blies (or  realistic  replicas  or  surrogates),  together  with  modeling  and  simulation, 
and  analysis  of  combat  data  which  will  provide  an  acceptable  assessment  of  the  vul- 
nerability of  the  system  as  configured  for  combat  will  be  performed." 

•  Require  that  a  waiver  to  the  requirement  for  testing  of  a  "complete  system  con- 
figured for  combat"  be  executed  prior  to  engineering  and  manufacturing  develop- 
ment, or  in  the  case  of  a  system  for  which  there  is  no  engineering  and  manufactur- 
ing development  phase,  at  the  time  a  decision  is  made  to  acquire  the  system. 

If  a  waiver  is  not  executed  in  accordance  with  the  time  Limits  established  above, 
testing  of  a  "complete  system  configured  for  combat"  may  be  waived  only  if  the  Sec- 
retary of  Defense,  without  delegation,  approves  the  waiver  and  provides  a  report  to 
Congress  indicating  a  waiver  was  granted. 

•  Add  to  the  definition  of  "realistic  vulnerabiUty  testing"  in  10  U.S.C.  2366(e)(3), 
after  the  phrase  "configured  for  combat",  the  phrase  "or  components,  subsystems, 
and  subassemblies  (or  realistic  repUcas  or  surrogates)  if  coupled  with  design  analy- 
ses, modeling  and  simulation,  and  analysis  of  combat  data,"  and  insert  the  word 
"complete"  prior  to  "configured." 

OPERATIONAL  TESTING 

Make  a  technical  change  in  10  U.S.C.  2399(a)  by  substituting  "major  system"  for 
"major  defense  acquisition  program"  each  place  it  appears,  substituting  in  para- 
graph (1)  "initial"  for  "an"  preceding  "operational  test  and  evaluation,"  and  sub- 
stituting "system"  for  "program."  Making  a  technical  change  in  the  title  of  para- 
graph (b)  by  adding  at  the  end  "of  Major  Defense  Acquisition  Programs." 

•  Allow  the  head  of  the  operational  test  and  evaluation  organization  of  the  mili- 
tary department,  if  the  military  department  is  the  milestone  decision-making  au- 
thority (ACAT  IC  programs),  to  approve  the  adequacy  of  operational  test  plans,  to 
analyze  the  results,  prepare  the  report  on  the  adequacy  of  tne  test  and  whether  the 
item  or  component  is  effective  and  suitable  for  combat,  submit  that  report  to  the 
appropriate  DOD  officials  and  to  Congress,  approve  proceeding  beyond  low-rate  ini- 
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tial  production,  approve  the  quantities  of  articles  that  must  be  tested,  and  waive 
the  use  of  certain  contractor  personnel  to  assist  in  testing. 

•  Amend  the  Secretary  of  Defense's  authority  to  modify  the  requirements  for 
operational  test  and  evaluation  where  the  Secretary  determines  that  such  testing 
would  be  unreasonably  expensive  or  impractical,  woidd  cause  an  unwarranted  delay, 
or  is  unnecessary  because  of  the  planned  acquisition  strategy,  and  describes  an  al- 
ternative method  to  assess  vulnerability/lethaUty  or  operational  suitability  and  ef- 
fectiveness to  provide  that  the  Secretary  would  submit  a  report  of  his  determination 
and  the  alternative  method  to  be  used  to  determine  operational  effectiveness  to  Con- 
gress for  major  defense  acquisition  programs  (as  defined  in  10  U.S.C.  2430)  vice  the 
current  requirement  to  certify  to  Congress  prior  to  modifying  the  requirement. 

•  Allow  the  Secretary  of  Defense  to  autnorize  the  use  of  contractor  employees  of 
the  contractor  whose  system  is  being  tested,  to  provide  analytic  and  logistic  support 
when  contractor  impartiaUty  is  assured  (add  to  10  U.S.C.  2399(d)). 

•  Allow  a  contractor  that  supported  developmental  test  analysis  to  support  oper- 
ational test  analysis  as  well,  when  impartiality  is  assured. 

•  In  neither  case,  may  the  Secretary  authorize  the  use  of  contractor  employees 
to  assist  in  the  establishment  of  criteria  for  data  collection,  criteria  for  performance 
assessment,  or  criteria  relating  to  the  evaluation  activities. 

•  Amend  10  U.S.C.  138(A)(2)(b)  to  limit  the  authority  of  the  Director  of  Oper- 
ational Test  to  designate  a  program  to  be  subject  to  the  testing  requirements  for 
Major  Defense  Acquisition  ft-ograms  (MDAPs)  that  would  otherwise  not  meet  the 
MDAP  threshold,  unless  the  program  is  so  interrelated  with  an  MDAP  that  testing 
of  the  programs  together  is  the  only  practical  means  to  assure  realistic  testing  of 
the  MDAP. 

Mr.  Chairman,  just  as  in  the  other  areas  we  have  targeted  for  acquisition  reform, 
we  need  to  take  a  good  hard  look  at  what  we  intend  to  accomplish  with  the  test 
and  evaluation  process  as  it  exists  today,  re-engineer  the  process  to  make  it  more 
efficient  and  cost  effective  given  the  changes  in  the  environment,  and  the  develop- 
ment of  new  enabling  technologies,  while  protecting  the  public's  interest.  A  lot  has 
changed  since  this  statute  was  enacted — most  important — ^the  Services'  view  of  the 
need  for  effective  testing. 

I  would  like  to  tell  you  that  there  is  imanimous  agreement  within  DOD  on  the 
changes  I  have  outlined  here  today.  But  if  I  did,  you'd  probably  be  concerned  that 
the  testing  community  had  given  away  too  much  in  the  process.  I  can  tell  you,  how- 
ever, that  the  recommendations  I  have  outlined  have  been  developed  after  many 
long  discussions  between  my  office  and  the  testing  commimity,  both  in  OSD  and  in 
the  Services,  and  they  reflect  a  genuine  effort  to  ensure  the  independence  and  effec- 
tiveness of  hve  fire  and  operational  testing  as  Congress  envisioned  it. 

I  truly  beheve  that  these  changes  will  simplify  the  testing  process  within  DOD, 
clarify  what  is  required,  and  strengthen  the  independent  live  fire  and  operational 
test  programs.  I  ask  only  that  you  give  us  a  chance  to  prove  we  can  execute  a  robust 
testing  program  with  the  changes  I  have  recommendea.  If  we  fail,  you  will  certjiinly 
have  the  ammunition  and  the  wherewithal  to  make  further  changes  in  the  statute 
at  that  time. 

I  will  be  happy  to  answer  any  questions  you  may  have. 


PREPARED  STATEMENT  OF  LOUIS  J.  RODRIGUES 

Mr.  Chairman  and  Members  of  the  Subcommittee: 

I  am  pleased  to  be  here  today  to  discuss  the  current  legislation  affecting  the  test 
and  evaluation  (T&E)  of  defense  systems  and  the  various  proposals  to  reform  that 
legislation.  As  you  know,  our  Office  has  a  long  historv  of  supporting  independent 
operational  T&E — conducted  as  early  as  possible  in  the  acquisition  process — as  a 
critical  management  control  over  the  process  to  acquire  defense  systems.  We  are 
currently  studying  the  Department  of  Defense's  (DOD)  use  of  the  low-rate  initial 
production  (LRIP)  concept  and,  in  particular,  the  advantages  and  disadvantages  of 
performing  at  least  some  operational  T&E  prior  to  starting  LRIP. 

Results  in  Brief 

Let  me  be  clear  right  up  front  that  GAO  generally  supports  most  of  the  acquisi- 
tion reform  proposals  being  debated  in  the  Congress.  However,  most  of  the  proposed 
revisions  to  the  current  T&E  legislation  should  not,  in  our  view,  be  enacted  because 
they  are  directed  at  perceived  rather  than  documented  problems  and  because  they 
would  undermine  a  key  management  control  over  the  system  acquisition  process.  In 
light  of  the  problems  that  we  continue  to  find  in  the  acquisition  defense  systems, 
the  priority  given  to  T&E  should  increase,  not  decrease,  as  would  occur  if  the  legis- 
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lative  proposals  were  enacted.  Our  reviews  of  test  and  evaluation  issues,  the  defense 
system  acquisition  process,  and  numerous  individual  weapon  programs  have  consist- 
ently shown  that  independent  operational  T&E  is  the  most  realistic  way,  short  of 
war,  to  determine  whether  DOD  is  getting  what  it  paid  for  in  each  system  acquisi- 
tion program.  We  also  believe  that,  if  a  program  is  "sold"  on  meeting  certain  re- 
quirements, DOD's  commitment  to  that  program  should  be  strictly  limited  until 
those  requirements  have  been  adequately  demonstrated.  Accordingly,  we  believe 
that  any  change  to  the  current  T&E  legislation  shoxild  at  least  preserve,  if  not 
strengthen  this  "fly-before-buy"  principle. 

Background 

Over  a  period  of  many  years.  Congress  has  been  greatly  concerned  about  the  per- 
formance of  weapon  systems  being  acquired  by  the  DOD.  As  early  as  1972,  Congress 
required  DOD  to  provide  it  with  information  on  the  operational  T&E  results  of 
major  weapons  systems  before  committing  to  production.  However,  Congress  contin- 
ued to  receive  reports  from  our  Office,  the  DOD  Inspector  General,  and  others  that 

(1)  weapon  systems  were  not  being  adequately  tested  before  beginning  production, 

(2)  fielded  systems  were  failing  to  meet  their  performance  requirements,  and  (3)  the 
operational  T&E  being  conducted  on  weapon  systems  was  of  poor  quality. 

In  an  effort  to  ensure  not  only  that  U.S.  military  personnel  receive  the  best  weap- 
on systems  possible  but  to  ensure  that  the  U.S.  government  receives  best  value  for 
the  defense  procurement  dollar,  the  Congress  enacted  legislation  which 

—defined  operational  T&Ei  (10  U.S.C.  139(a)(2)(A));  and 

— established  the  office  of  the  Director,  Operational  Test  and  Evaluation 
(DOT&E),  and  assigned  specific  oversight  duties  and  responsibilities  (10  U.S.C. 
139(a)(1)). 

The  Congressional  Military  Reform  Caucus  convened  a  panel  in  1989  to  review 
the  conduct  of  operational  and  live  fire  testing,  and  how  DOD  was  implementing 
the  testing  laws.  The  panel  found  that  top  DOD  decisionmakers  needed  to  pay 
greater  attention  and  commitment  to  operational  T&E.  In  the  House  Armed  Serv- 
ices Committee  report  accompanying  the  National  Defense  Authorization  Act  for 
Fiscal  Years  1990-1991,  the  Committee  noted  its  concern  that  DOD  was  procuring 
large  quantities  of  weapons  systems  without  completing  operational  testing.  The 
Phoenix  missile  was  cited  as  an  example  where  over  52  percent  of  the  program 
quantity  had  been  procured  before  operational  testing  was  completed. 

As  a  result  of  these  and  other  cases.  Congress  passed  additional  T&E  legislation 
which,  among  other  things, 

— required  that  a  major  system  2  may  not  proceed  beyond  LRIP  until  its  initial 
operational  T&E  is  completed  (10  U.S.C.  2399(a)); 

— specified  that-operational  T&E  of  a  major  defense  acquisition  program  3  may  not 
be  conducted  until  the  DOT&E  approves  the  adequacy  of  the  plans  for  operational 
T&E  (10  U.S.C.  2399(b)(1)); 

— specified  the  purposes  of  LRIP,  the  timing  of  the  determination  of  quantities  to 
be  procured  in  LRIP,  and  provided  for  certain  exemptions  to  the  LRIP  requirements 
(10  U.S.C.  2400); 

— required  that  DOT&E  analyze  the  results  of  the  operational  T&E  conducted  for 
each  major  defense  acquisition  program  and,  prior  to  a  final  decision  to  proceed  be- 
yond LRIP,  report  on  the  adequacy  of  the  testing  and  whether  the  resiilts  of  such 
T&E  confirm  that  the  items  tested  are  effective-*  and  suitable 5  for  combat  (10 
U.S.C.  2399(b)(2)  and  (b)(4)); 


iThe  term  "operational  test  and  evaluation"  means  (1)  the  field  test,  under  realistic  condi- 
tions, of  any  item  of  (or  key  component  of)  weapons,  equipment,  or  munitions  for  the  purpose 
of  determining  the  effectiveness  and  suitability  of  the  weapons,  equipment,  or  munitions  for  use 
in  combat  by  typical  military  users;  and  (2)  the  evaluation  of  the  results  of  such  test. 

2  According  to  DOD  regulation,  a  major  system  is  defined  as  a  system  whose  research,  devel- 
opment, test,  and  evaluation  cost  is  estimated  to  exceed  $115  million  in  fiscal  year  1990  dollars 
or  procurement  cost  is  estimated  to  exceed  $540  million  in  fiscal  year  1990  dollars. 

3  A  major  defense  acquisition  program  is  defined  as  a  system  whose  research  and  development 
cost  is  expected  to  exceed  $300  million  in  fiscal  year  1990  dollars  or  procurement  cost  is  ex- 
pected to  exceed  $1.8  billion  in  fiscal  year  1990  dollars. 

4  DOD  defines  operational  effectiveness  as  "the  overall  degree  of  mission  accomplishment  of 
a  system  when  used  by  representative  personnel  in  the  environment  planned  or  expected  (e.g., 
natural,  electronic,  threat,  etc.)  for  operational  employment  of  the  system  considering  organiza- 
tion, doctrine,  tactics,  survivability,  vulnerability,  and  threat  (including  countermeasures,  initiad 
nuclear  weapons  effects,  nuclear,  biological,  and  chemical  contamination  (NBCC)  threats)." 

6  DOD  defines  operational  suitability  as  "the  degree  to  which  a  system  can  be  placed  satisfac- 
torily in  field  use  with  consideration  given  to  availability,  compatibility,  transportability,  inter- 
operability, reliability,  wartime  usage  rates,  maintainability,  safety,  human  factors,  manpower 
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— specified  that,  for  ptirposes  of  proceeding  beyond  LRIP,  required  operational 
T&E  not  include  an  operational  assessment  based  exclusively  on  computer  model- 
ing, simulation,  or  certain  analyses  (10  U.S.C.  2399(h)(1));  and 

— specified  that,  for  certain  covered  systems  and  major  munition  or  missile  pro- 

f rams  6  ,  realistic  survivability  and  lethality  T&E,  respectively,  be  conducted;  any 
esign  deficiency  identified  by  the  testing  be  corrected;  and  specific  reports  be  sub- 
mitted before  proceeding  beyond  LRIP  (10  U.S.C.  2366). 

Section  800  of  the  National  Defense  Authorization  Act  for  Fiscal  Year  1991  di- 
rected the  Under  Secrettuy  of  Defense  for  Acquisition  (USD-A)'  to  appoint  an  advi- 
sory panel  of  government  and  private  sector  experts  to  start  the  process  of 
rationalizing,  codifying,  and  streamlining  the  laws  underpinning  the  defense  acaui- 
sition  system.  The  Panel  was  to  review  all  laws  affecting  DOD  procurement  and  to 
issue  a  report  for  transmission  by  the  Secretary  of  Defense  to  Cfongress.  The  report 
was  to  contain  recommendations  to  eliminate  any  laws  "unnecessary  for  the  estab- 
Ushment  and  administration  of  the  buyer  and  seller  relationship  in  procurement"; 
ensure  the  "continuing  financial  and  ethical  integrity  of  defense  procurement  pro- 
grams"; and  "protect  the  best  interests  of  the  Department  of  Defense." 

Issued  in  January  1993,  the  Panel's  report  8  and  recommendations  address  those 
statutes  governing  (1)  DOD's  procurement  of  commercial  items;  (2)  simplified  acqui- 
sition thresholds  and  socioeconomic  poUcies;  (3)  contract  formation  and  management 
(such  as  truth  in  negotiations,  protests,  and  cost  accounting  standards);  (4)  defense 
technology  and  industrial  base  matters;  (5)  intellectual  property  rights  and  stand- 
ards of  conduct;  and  (6)  major  systems,  testing,  and  other  matters. 

We  recently  reported  »  on,  among  other  things,  the  Section  800  Panel's  compliance 
with  certain  legislative  requirements.  Also,  our  Office  has  recently  addressed  other 
aspects  of  acquisition  reform  at  a  joint  hearing  of  the  Senate  Conmiittee  on  Armed 
Services  and  the  Senate  Committee  on  Governmental  Affairs.  Our  discussion  here 
today  will  be  limited  to  the  proposals  to  revise  the  T&E  legislation. 

Current  Proposals  to  Revise  T&E  Legislation 

The  Section  800  Panel  made  several  legislative  proposals  in  the  T&E  area  that 
have  also  been  included  to  various  extents  in  the  pending  Senate  and  House  bills 
(S.  1587  and  H.R.  3586)  and  the  Conyers/Clinger  proposals  to  reform  the  defense 
acquisition  system.  The  major  T&E  proposals  currently  imder  consideration  seek  to: 

(1)  authorize  the  use  of  component,  subsystem,  and  subassembly  testing  instead 
of  fiill-up  system  lo  testing  in  determinine  the  survivability  of  "high  value"  systems; 

(2)  provide  authority  to  wmve  survivability  testing  after  full-scale  development  be- 
gins; 

(3)  provide  for  the  use  of  "different"  operational  T&E  procedures  in  certam  cir- 
cumstances; and 

(4)  provide  an  exception  to  the  LRIP  requirements  for  "strategic  defense  missile 
systems." 

GAO  Assessment  of  Legislative  Proposals 

We  reviewed  the  T&E  proposals,  the  Section  800  report,  and  all  available  support- 
ing materials  with  the  following  questions  in  mind: 

(1)  Is  the  current  T&E  legislation  a  sensible  approach  to  address  deficiencies  in 
the  system  acquisition  process? 

(2)  To  what  extent  is  the  current  T&E  legislation,  as  implemented,  effective  in 
meeting  its  objectives? 

(3)  Does  the  current  T&E  legislation  impose  extraordinary  penalties  or  burdens 
on  DOD? 

As  I  will  discuss  in  some  detail  below,  we  have  concluded  that  the  justifications 
for  the  "r&E  proposals  do  not  answer  these  questions  and  that  the  proposals  appear 
to  be  directed  at  perceived  rather  than  documented  problems.  We  believe  the  pro- 


supportability,  logistics  supportability,  natural  environmental  effects  and  impacts,  documenta- 
tion, and  training  requirements." 

6 10  U.S.C.  2366  contains  definitions  of  the  terms  "covered  system"  and  "major  mumtions  pro- 
gram." For  example,  it  defines  covered  system  as  a  vehicle,  weapon  platform,  or  conventional 
weapon  system  that  (a)  includes  features  designed  to  provide  some  degree  of  protection  to  users 
in  combat  and  (b)  is  a  major  system. 

'Now  the  Under  Secretary  of  Defense  for  Acquisition  and  Technology  (USD-A&T). 

^  Streamlining  Defense  Acquisition  Laws;  Report  of  the  Acquisition  Law  Advisory  Panel. 

0 Acquisition  Reform:  DOD  Acquisition  Law  Advisory  Panel's  Operations  and  Report  (GAO/ 
NSIAD-94-5.  Dec.  1,  1993).  ^     „     ^.  ^  . 

10  The  term  "full-up  system"  is  synonymous  with  the  term  "configured  for  combat,  which  is 
defined  in  10  U.S.C.  2366(e)  (5)  as  a  weapon  system,  platform,  or  vehicle  loaded  or  equipped 
with  all  dangerous  materials  (including  flammables  and  explosives)  that  would  normally  be  on 
board  in  combat. 
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posals  to  modify  the  T&E  legislation  would  dilute  the  effectiveness  of  T&E  as  a  key 
management  control  and  thus,  would  weaken  rather  than  strengthen  the  system  ac- 
quisition process.  Therefore,  we  would  not  support  the  major  T&E  proposals  and 
strongly  suggest  that  they  be  deleted  from  the  acquisition  reform  bills  pending  be- 
fore the  House  and  Senate. 

Although  independent  T&E  is  clearly  the  most  realistic  way,  short  of  war,  to  de- 
termine if  DOD  is  getting  what  it  paid  for,  T&E  is  eenerallv  viewed  by  the  acquisi- 
tion community  as  a  requirement  imposed  by  outsiders  rather  than  a  management 
tool  to  identify,  evaluate,  and  reduce  technical  risks  and,  therefore,  a  means  to  more 
successful  programs.  In  that  light,  the  Panel's  report  stated  that,  in  some  cases,  the 
developers  express  frustration  over  the  delays  and  expense  imposed  on  their  pro- 
grams by  overzealous  testers.  We  would  point  out,  however,  that  the  testers  can  test 
only  what  is  called  for  in  the  test  and  evaluation  master  plan,  which  is  put  together 
by  the  developers,  users,  and  testers.  If  the  developers  promise  to  develop  a  system 
with  specific  capabilities,  it  is  incumbent  on  them  to  clearly  demonstrate  those  capa- 
bilities during  the  testing  program.  To  the  extent  that  they  cannot  do  that,  we  be- 
lieve it  is  unfair  and  inappropriate  for  the  developers  to  consider  the  testers  "over- 
zealous"  simply  because  they  may  be  the  bearer  of  the  bad  news. 

The  legislative  proposals  also  seem  to  support  the  philosophy  that  we  encounter 
too  frequently  at  the  program  office  level — that  it  is  most  important  to  get  the  pro- 
gram into  production  despite  the  uncertainty  that  the  system  will  work  as  proniised 
or  intended.  Also,  an  underlying  assumption  seems  to  be  that  a  key  acquisition 
issue  is  to  reduce  the  length  of  the  system  acquisition  process  and  an  obvious  way 
to  shorten  the  process  is  to  reduce  troublesome  parts  of  the  process,  such  as  testing. 
Moreover,  we  have  seen  numerous  instances  where  DOD  has  chosen  not  to  promptly 
correct  problems  identified  during  testing  and  decided  to  proceed  into  production 
and  deployment  despite  the  identified  problems.  With  that  in  mind,  we  do  not  be- 
lieve it  would  be  in  the  best  interests  of  the  U.S.  government  to  weaken  the  testing 
requirements  and  discipline  embodied  in  the  current  legislation. 

***** 

I  will  now  address  the  individual  proposals  in  the  order  mentioned  earlier. 

(1)  Authorize  the  use  of  component,  subsystem,  and  subassembly  testing  instead  of 
full-up  system  testing  in  determining  the  survivability  of  "high  value"  systems. 

The  Section  800  Panel  concluded  that  full-up  system  testing  can  add  unnecessary 
time  and  expense  over  component  testing  and  that  the  provision  in  the  current  leg- 
islation permitting  waivers  from  fuUup  svstem  testing  was  not  adequate.  However, 
in  creating  an  exemption  for  all  "high  value"  systems,  the  proposal  would  eUminate 
reporting  to  the  Congress  on  plans  for  alternatives  to  full-up  survivability  testing, 
as  is  required  under  the  current  waiver  process.  Moreover,  the  fact  that  "high  value" 
is  not  defined  is  an  added  concern  since  more  systems  may  be  aflFected  than  Con- 
gress may  anticipate. 

In  addition,  the  Panel's  conclusion  admittedly  did  not  consider  the  results  of  a 
January  1993  National  Research  Council  study. n  Among  other  things,  the  Council 
concluded  that  the  existing  live  fire  test  law  was  a  valuable  contribution  to  vulner- 
ability assessment  and  to  the  design  of  survivable  aircraft.  The  Council  also  con- 
cluded that  the  law  was  satisfactory  in  its  present  form,  including  its  mandate  for 
full-up  system  testing,  because  the  waiver  process  permitted  exceptions  where  ap- 
propriate, i'he  Council,  however,  recommended  that  DOD  formalize  the  waiver  proc- 
ess by  developing  a  risk-benefit  assessment  methodology  that  could  be  used  uni- 
formly to  determine  whether  a  full-scale,  full-up  system  test  program  for  any  par- 
ticular aircraft  is  "unreasonably  expensive  and  impractical." 

In  light  of  the  Council's  study  results  and  our  earlier  work  in  this  area,i2  we  rec- 
ommend that  Congress  not  change  the  legislation  to  authorize  less  than  full-up  sys- 
tem survivability  testing  for  "hi^  value"  systems.  A  case-by-case  assessment  of  the 
costs  and  benefits  of  full-up  system  testing—which  is  the  current  practice — is  pref- 
erable to  an  across-the-board  exemption  for  "high  value"  systems. 

(2)  Provide  authority  to  waive  survivability  testing  after  full-scale  development  be- 
gins. 

The  Section  800  Panel  stated  that  this  provision  was  needed  because  the  cost  and 
complexity  of  survivability  testing  may  not  be  known  until  after  some  systems  enter 


11  This  study,  entitled  Vulnerability  Assessment  of  Aircraft:  A  Review  of  the  Department  of  De- 
fense Live  Fire  Test  and  Evaluation  Program,  was  requested  by  DOD  in  an  attempt  to  resolve 
the  controversy  regarding  the  law's  requirements  and  to  obtain  an  independent  opinion  regard- 
ing the  total  vulnerability  assessment  process  and  the  law's  contribution  to  that  process. 

izSee  Live  Fire  Testing:  Evaluating  DOD's  Programs  (GAO/PEMD-87-17,  Aug.  17,  1987). 
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full-scale  development.  We  do  not  see  the  need  for  this  provision  because  the  surviv- 
ability test  plan  should  be  completed  and  any  basis  for  a  waiver  from  full-up  system 
testing  should  be  evident  prior  to  the  start  of  ftill-scale  development.  As  a  reminder, 
the  survivability/lethality  test  law  requires  consideration  of  survivability/lethality 
during  the  system  development  phase.  The  proposal  to  allow  waivers  later  in  the 
system  acquisition  process  may  give  program  officials  an  incentive  to  delay  planning 
and  implementing  the  tests  as  long  as  possible  and  then  apply  for  a  waiver. 
We  believe  much  more  attention  needs  to  be  focused  on  laentifying  and  addressing 

{)roblem  areas  earlier  in  the  system  acquisition  process — ^through  survivability, 
ethality,  and  operational  testing — ^because  early  fixes  are  less  expensive,  easier  to 
implement,  and  less  disruptive  to  the  program.  In  the  survivabili^  test  area,  more 
attention  needs  to  be  placed  on  how  to  do  the  testing  in  a  timely,  efficient  and  effec- 
tive manner  and  less  attention  on  how  to  obtain  a  waiver  from  the  testing. 

(3)  Provide  for  the  use  of  "different"  operational  T&E  Drocedures  in  certain  cir- 
cumstances . 

This  proposal  is  the  most  troublesome  to  us  because  it  is  ambiguous  and  poten- 
tially far-reaching.  The  alternative  testing  procedures  would  be  permitted  if  the  nor- 
mal operational  T&E  procedures  were  found  to  (a)  be  unreasonably  expensive  and 
impractical,  (b)  cause  unwarranted  delay,  or  (c)  be  unnecessary  because  of  the  acqui- 
sition strategy  for  that  system.  We  £u-e  very  concerned  about  the  potential  impact 
of  this  proposal  and  strongly  recommend  that  it  not  be  enacted. 

First,  until  shown  otherwise,  we  believe  that  "regular"  operational  T&E  (i.e.  field 
testing  under  realistic  conditions)  is  the  best  way,  by  far,  to  ensure  that  an  item 
will  be  effective  and  suitable.  Second,  the  proposal  does  not  specify  what  is  meant 
by  "different"  operational  T&E  procedures.  For  example,  would  computer  modeling 
and  simulation,  paper  studies,  etc.,  be  substituted  for  field  tests?  We  want  to  em- 
phasize here  that  the  ultimate  objective  of  operational  T&E  is  to  clearly  dem- 
onstrate the  system's  operational  effectiveness  and  suitabUity  in  a  realistic  environ- 
ment. There  is  no  prohibition  on  the  use  of  computer  modeling  and  simulation  as 
a  supplement  to  operational  T&E.  Computer  modeling  and  simulation,  as  well  as 
other  developmental  tools,  should  be  used  to  evaluate  the  operational  utility  of  the 
system  and  to  identify  key  aspects  of  the  system  design  to  be  addressed  in  oper- 
ational T&E.  Lastly,  this  proposal  would  eliminate  the  role  of  DOT&E  in  approving 
the  test  plans  before  operational  testing  begins. 

In  addition,  we  question  the  prescribed  bases  for  the  use  of  "different"  operational 
T&E,  as  follows: 

— Where  "regular"  operational  T&E  is  unreasonably  expensive  and  impractical. 
Fundamentally,  we  are  hard-pressed  to  envision  situations  where  "regular"  oper- 
ational tests  have  been  or  could  be  "unreasonably  expensive  and  impractical"  when 
judged  against  the  potential  expenditures  of  many  millions,  if  not  billions,  of  dollars 
for  a  weapon  system.  Moreover,  we  point  to  the  fact  that,  under  the  current  proce- 
dures, the  operational  testing  to  be  conducted  pnd  the  testers'  role  in  each  program 
is  specified  in  the  test  and  evaluation  master  plan,  a  document  put  together  by  the 
developers,  users,  and  testers.  We  believe  the  current  processes  to  develop  and  ap- 
prove the  test  plans  provide  adequate  opportunity  to  structure  affordable  and  prac- 
tical programs  of  "regular"  operational  T&E.  Unfortunately,  this  provision  would 
provide  the  opportunity  for  the  svstem  developers  to  circumvent  these  processes. 

— Where  "regular"  operational  T&E  woula  cause  unwarranted  delay.  Although 
operational  T&E  takes  time,  one  should  look  at  the  causes  for  delays  attributed  to 
testing — ^for  example,  are  they  due  to  excessive  or  imnecessary  testing  or  to  an  inad- 
equate system  struggling  to  meet  its  gO£ils?  While  there  are  many  potential  causes 
of  program  delays,  our  experience  is  that  programs  are  often  delayed  or  disrupted 
because  they  did  not  do  well  in  development  and  testing.  In  addition,  we  often  find 
that  system  acquisition  strategies  frequently  do  not  allow  enough  time  for  resolution 
of  problems  expected  to  be  found  during  testing.  Therefore,  we  oelieve  that  this  pro- 
vision will  not  improve  the  system  acquisition  process  because  it  would  provide  the 
opportunity  to  single  out  T&E  as  the  cause  of  program  delavs. 

Where  "regular"  operational  T&E  would  be  unnecessary  because  of  the  acquisition 
strategy  for  that  system.  This  proposal  is  very  broad  but,  according  to  the  Section 
800  Panel  report,  is  directed,  at  least  in  part,  at  situations  involving  the  acquisition 
of  non-developmental  items.  Regardless  of  whether  items  are  developmental  or  non- 
developmental,  the  items  should  demonstrate  that  they  meet  military  requirements 
before  being  procured.  We  have  seen  several  cases — such  as  the  Heavy  Equipment 
Transporter  System,  the  Family  of  Medium  Tactical  Vehicles,  and  the  Palletized 
Loading  System — where  operational  T&E  has  shown  that  a  "non-developmental 
item"  was  not  fully  effiective  and/or  suitable  for  miUtary  operations. 

(4)  Provide  an  exception  under  the  LRIP  requirements  for  "strategic  defense  missile 
systems." 
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The  Section  800  Panel's  rationale  for  this  proposal  is  that  these  systems  have  low 
production  quantities.  That  rationale  is  not  necessarily  correct  in  that  at  least  some 
of  these  programs  may  involve  production  of  dozens,  if  not  hundreds,  of  units. 
Therefore,  these  types  of  systems  should  be  expected  to  have  a  phase  of  low-rate 
production  followed^  by  full-rate  production.  Naval  ships  and  satellites  have  been 
given  special  legislative  treatment  because  they  do  not  typically  ramp-up  to  full-rate 
production.  Therefore,  we  do  not  see  any  justification  for  exempting  "strategic  de- 
fense missile  systems"  from  the  LRIP  legislation. 

***** 

We  believe  that  simpler,  more  flexible,  and  more  responsive  procedures  in  the 
testing  area  should  not  compromise  the  necessary  discipline  embodied  in  the  current 
legislation.  We  have  not  foimd,  and  the  Section  800  Panel  has  not  provided  the 
analysis  to  show  that  T&E  is  one  of  the  problems  with  the  system  acquisition  proc- 
ess. On  the  contrary,  ovu-  large  body  of  work  indicates  that  the  testing  requirements 
may  need  to  be  strengthened  and  expanded. 

Ongoing  GAO  Work  on  Low-Rate  Initial  Production 

For  many  major  and  non-major  system  acquisition  programs,  LRIP  has  been  per- 
mitted to  begin  and  continue  based  not  on  the  systems'  technical  maturity  but  on 
schedule  considerations.  For  example,  the  Advanced  Medium  Range  Air-to-Air  Mis- 
sile, the  B-IB,  the  C-17,  the  Short-Range  Unmanned  Aerial  Vehicle,  the  B-2,  and 
numerous  electronic  warfare  programs  clearly  started  production  prematurely  and 
with  Uttle,  if  any,  indication  ot  the  system's  operational  effectiveness  and  suitability. 
The  current  LRIP  legislation  permits — and  tends,  in  our  view,  to  encourage — DOD 
to  start  LRIP  before  any  operational  testing  is  conducted. 

Many  of  the  systems  that  enter  LRIP  prematurely  later  experience  significant  ef- 
fectiveness and/or  suitability  problems  during  operational  testing  that  require  major 
design  changes  to  correct  and  additional  testing  to  verify.  In  those  cases  where  pro- 
duction is  well  underway,  costly  retrofits  are  required  for  delivered  units.  For  exam- 
ple, the  C-17  and  T-45A  aircraft,  the  B-IB  defensive  avionics,  the  Advanced  Me- 
dium Range  Air-to-Air  Missile,  and  numerous  electronic  warfare  systems  needed 
many  design  changes  and  costly  retrofits  because  of  poor  test  results.  In  some  cases, 
such  as  the  B-IB  defensive  avionics,  the  problems  identified  could  not  be  corrected, 
resulting  in  deficient  systems  being  fielded. 

Moreover,  even  though  LRIP  may  have  been  started  primarily  to  provide  articles 
for  testing,  DOD  very  often  continues  production  beyond  that  purpose,  despite  tech- 
nical problems  identified  during  operational  testing.  For  example,  despite  a  number 
of  serious  technical  problems,  as  many  as  72  T-45A  aircraft  of  the  total  program 
quantity  of  268  will  be  authorized  under  LRIP.  In  addition,  significant  quantities 
of  several  electronic  warfare  systems  were  procured  despite  significant  technical 
problems  and  shortcomings. 

To  reduce  the  risk  of  finding  major  operational  effectiveness  and  suitability  prob- 
lems after  production  starts,  we  have  regularly  recommended  (1)  less  concurrent  de- 
velopment and  production,  and  (2)  as  much  operational  testing  as  possible  before 
production  starts.  Nevertheless,  we  continue  to  find  that  highly  concurrent  acquisi- 
tion strategies  are  featured  in  many  current  major  and  nonmajor  programs,  with 
little,  if  any,  operational  testing  expected  until  well  after  the  start  of  LRIP.  For  ex- 
ample, the  F/A-18E/F  aircraft  and  F-22  aircraft  programs  plan  to  make  substantial 
commitments  to  production  before  completing  initial  operational  T&E.  Contrary  to 
general  commercial  practices  and  despite  the  risk  of  being  committed  to  producing 
deficient  systems,  defense  system  acquisition  programs  continue  to  enter  and  pro- 
ceed well  into  production  before  being  put  under  serious  scrutiny.  In  today's  na- 
tional security  environment,  we  believe  there  should  be  very  few  cases  where  there 
is  a  need  to  assume  the  additional  risks  inherent  in  a  highly  concurrent  acquisition 
strategy. 

Based  on  its  review  of  major  defense  acquisition  programs, i3  the  DOD  Inspector 
General  also  found  that  programs  were  entering  LRIP  without  meeting  the  develop- 
ment, testing,  and  production  readiness  prerequisites  and  recommended  that  pro- 
gram-specific exit  criteria  be  set  up  to  ensure  that  those  prerequisites  be  met  before 
entering  LRIP.  Because  of  the  costs  involved  and  the  inability  or  unwillingness  to 
curtail  production  after  it  starts,  we  agree  that  controls  are  urgently  needed  over 
the  start  and  continuation  of  LRIP.  Most  importantly,  we  beUeve  that  a  reauirement 
to  conduct  at  least  a  phase  of  operational  testing  before  LRIP  starts  would  be  fea- 


i^Low-Rate  Initial  Production  in  Major  Defense  Acquisition  Programs  (Report  No.  94-014; 
Nov.  9,  1993). 
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sible  in  most  cases  and  would  be  an  effective  management  control  over  the  pre- 
mature start  of  production. 

The  requirement  in  the  National  Defense  Authorization  Act  for  Fiscal  Year  1994 
that  the  Secretary  of  Defense  ensure  appropriate,  rigorous,  and  structured  testing 
be  completed  prior  to  LRIP  of  any  electronic  combat  or  command,  control,  and  com- 
munication countermeasure  system  is  definitely  a  move  in  the  right  direction.  How- 
ever, its  provisions — as  are  most  legislative  provisions  on  T&E  matters — are  appli- 
cable only  to  major  defense  acouisition  programs.i'*  We  presently  ejcpect  no  more 
than  three  or  four  systems  will  be  coverM  by  this  provision.  Our  findings — particu- 
larly evident  in  the  electronic  warfare  and  command,  control  and  communications 
area  but  common  on  other  system  developments — are  not  Limited  to  major  systems 
or  major  defense  acquisition  programs.  In  fact,  some  of  these  problems  appear  to 
be  more  acute  with  the  non-mayor  programs.  We  feel  strongly  that  the  principles, 
if  not  the  reporting  requirements,  oi  the  current — and  we  hope  strengthened — T&E 
legislation  should  be  applicable  to  non-major  as  well  as  major  systems. 

Potential  Impact  of  Acquisition  "Culture" 

In  late  1992,  we  reported  is  on  the  problems  in  acquiring  defense  systems  and  de- 
scribed the  entrenched  "culture"  that  has  so  effectively  resisted  many  attempts  at 
reforming  the  system  acquisition  process.  We  continue  to  see  that  culture  hard  at 
work  today  in  this  attempt  to  significantly  weaken  the  current  legislation  control- 
ling T&E.  We  do  not  support  the  proposed  changes  because  they  would  tend  to 
weaken  the  primary  mechanism  that,  m  our  view,  can  be  used  effectively  to  dis- 
cipline the  system  acquisition  process. 

Plain  and  simple,  we  support  increased  accountability  in  the  system  acquisition 
process.  We  continue  to  believe  that,  if  a  program  is  "sold"  on  meeting  certain  re- 
quirements, DOD's  commitment  to  that  program  should  be  strictly  Umited  until 
tnose  requirements  have  been  adequately  demonstrated.  Based  on  the  interim  re- 
sults of  our  ongoing  work,  we  believe  that  DOD's  initial  commitment  to  LRIP  of  a 
weapon  system  should  not  be  made  until  developmental  and  operational  test  results 
show  that  the  system  (a)  is  potentially  operationally  effective,  (b)  will  not  need  cost- 
ly or  extensive  changes,  and  (c)  has  a  reasonable  chance  to  successfully  complete 
its  remaining  operational  T&E  in  a  timely  fashion.  We  expect  to  complete  our  ongo- 
ing work  by  the  late  summer. 

Most  participants  in  the  defense  system  acquisition  process  ascribe  at  least  in 
principle  to  a  "^fly-before-buy"  acquisition  philosophy.  However,  many  of  the  current 
T&E  proposals  would  in  practice  move  DOD  away  fi-om  that  principle. 

***** 

In  response  to  Senator  Roth's  specific  question  regarding  the  current  organiza- 
tional placement  of  the  Office  of  the  Director,  Operational  Test  and  Evaluation 
(DOT&E),  we  have  found  that  the  organization  chart  contained  in  the  Secretary  of 
Defense's  January  1994  report  to  the  Congress  to  be  in  error.  That  report  erro- 
neously shows  DOT&E  reporting  directly  to  the  Under  Secretary  of  Defense  for  Ac- 
quisition rather  than  directly  to  the  Secretary  of  Defense.  At  this  time,  DOT&E  con- 
tinues to  report  directly  to  the  Secretary  of  Defense.  However,  the  National  Defense 
Authorization  Act  for  Fiscal  Year  1994  deleted  the  legislative  requirement  that 
DOT&E  report  directly,  without  intervening  review  or  approval,  to  the  Secretary  of 
Defense.  The  conferees  suggested  that  DOT&E  be  placed  under  the  Comptroller's 
Office. 

Mr.  Chairman,  that  concludes  my  statement.  I  will  be  happy  to  answer  any  ques- 
tions you  or  members  of  the  Subconunittee  may  have. 


Prepared  Statement  of  Russell  Murray 

Mr.  Chairman,  I  am  both  delighted  and  greatly  honored  to  appear  before  you  and 
this  distinguished  Subcommittee  on  a  subject  of  such  great  importtmce:  operational 
testing. 

I  would  Uke  to  speak  to  you  today  about  what  seem  to  me  in  retrospect  to  be  some 
of  major  policy  issues  in  operationsd  testing. 

Operational  Testing  and  Independence 


i*The  legislation  in  question  here  refers  to  acquisition  category  I  (ACAT  I)  programs,  which 
are  also  megor  defense  acquisition  programs. 

^^  Weapons  Acquisition:  A  Rare  Opportunity  for  Lasting  Chance  (GAO/NSIAD-93-15,  Dec. 
1992). 


46 

The  first  issue  is  whether  the  Director  of  Operational  Testing  should  be  independ- 
ent of  the  Acquisition  Executive — in  broader  terms,  whether  the  operational  testers 
of  weapons  should  be  independent  of  the  technological  developers  of  weapons. 

Acquisition  Executives  tend  to  oppose  such  independence.  They  argue  that,  as  Ac- 
quisition Executives,  they  Eire  of  course  responsible  for  the  creation  of  all  new  weap- 
ons. To  carry  out  that  responsibility,  they  must  have  authority  over  all  phases  of 
that  creation.  Because  one  of  those  phases  is  testing,  and  because  testing  includes 
operational  testing,  they  must  have  authority  over  the  Director  of  Operational  Test- 
ing. 

Wrong. 

That  manifesto  seriously  prejudices  the  balance  of  influence  between  the  weapon 
makers  (represented  by  these  Acquisition  Executives),  emd  the  weapon  users  (rep- 
resented by  the  Directors  of  Operational  Testing). 

Without  independent  operational  testing  to  weigh  in  on  the  side  of  the  weapon 
users,  the  weapon  makers'  side  of  the  scales  is  too  heavy  with  money,  and  with  poli- 
tics, and  with  constituencies.  Without  independent  operational  testers,  there's  noth- 
ing on  the  other  side  of  the  scales  but  the  weapon  users  out  in  the  field — ^the  fighter 
pilot,  the  crew  chief,  the  platoon  leader,  the  gunner's  mate.  In  spite  of  their  num- 
bers, these  weapon  users,  far  from  Washington's  corridors  of  power,  cannot  by  them- 
selves match  the  influence  of  the  weapon  makers. 

•  The  fighter  pilot's  no  match  for  the  wealthy  and  influential  military  con- 
tractors who  want  to  get  their  production  lines  rolling  as  soon — and  as  prolifi- 
cally — as  they  can. 

•  The  crew  chiefs  no  match  for  the  hundreds  of  thousands  of  those  contrac- 
tors' employees  in  dozens  of  States  who  imderstandably  lean  on  Washington  to 
look  out  for  their  economic  livelihood. 

•  The  platoon  leader's  no  match  for  the  scientists  and  engineers  whose  rep- 
utations could  be  in  trouble  if  their  promised  technological  breakthroughs  fail 
in  operational  testing. 

•  And  the  gunner  mate's  no  match  for  the  managers  at  all  levels  whose  job 
is  to  get  their  new  weapons  deployed  on  schedule. 

In  short,  what  the  weapon  users  out  in  the  field  are  no  match  for  is  the  weapon 
makers'  great  coalition  oi  researchers  and  developers  and  manufacturers  who  dwell 
within  the  Acquisition  Executive's  sphere  of  influence.  The  danger  in  that  imbalance 
of  influence  is  that  new  weapon  projects  can  then  grow  into  avalanches,  sweeping 
away  what  little  opposition  there  may  be  in  their  path. 

What  's  needed  is  a  counterbalance — some  way  of  making  sure  that  the  weapon 
users'  interests  are  not  swept  away  by  that  avalanche.  The  users  counterbalance  is 
independent  operational  testing. 

But  why  independent?  Instead  of  reporting  to  the  Secretary  of  Defense,  why 
shouldn't  the  Director  of  Operational  Testing  report  to  the  Acquisition  Executive, 
who  would  then  balance  the  competing  interests  of  the  weapon  makers  and  the 
weapon  users? 

The  fact  that  we  have  yet  to  nominate  a  professional  warrior  to  be  the  Acquisition 
Executive — and  probably  never  will — suggests  the  weakness  in  that  proposition.  The 
reason  lies  in  what  we  demand  (or  should)  of  any  prospective  Acquisition  Executive. 
To  handle  that  job  (already  felt  by  some  to  be  impossible)  we  demand  superb  techno- 
logical training  and  competence,  an  outstanding  record  in  research  and  develop- 
ment, proven  skill  in  industrial  management,  and  experience  in  managing  Defense 
acquisition  because  it's  no  job  for  a  Pentagon  neophyte. 

But  if,  in  addition  to  all  that,  we  also  say  that  the  Acquisition  Executive  must 
be  a  professional  warrior,  a  person  with  the  background  and  experience  in  the  field 
to  instinctively  understand  what  a  weapon  user  needs,  then  we're  almost  certainly 
demanding  the  impossible. 

In  summary,  to  ensure  a  proper  balance  between  the  makers  and  the  users  of 
weapons,  the  Acquisition  Executive  and  the  Director  of  Operational  Testing  must 
be  independent.  "The  qualifications  we  should  demand  of  two  positions  are,  in  effect, 
mutually  exclusive.  The  Director  of  Operational  Testing  should  be  a  creature  of  the 
community  that  uses  rather  than  creates  weapons.  And,  as  recommended  24  years 
ago  by  the  Fitzhugh  Blue  Ribbon  Panel,  the  Director  should  report  directly  to  the 
Secret£iry  of  Defense. 

Conceptual  Operational  Testing 

A  second  policy  question  is  when  in  the  development  process  should  operational 
testing  occur.  I  sense  that  operational  testing  is  increasingly  being  viewed  as  little 
more  than  a  "final  exam".  Though  that  is  an  important  milestone,  operational  test- 
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ing  can  be  even  more  important  at  the  very  inception  of  a  project.  Charles  Myers' 
testimony  before  this  Committee  on  June  23,  1983  illustrates  the  point.  He  said: 

"The  question  of  [visual]  acquisition  of  tanks  seemed  important  relative  to 
being  able  to  use  a  Maverick  missile.  So  we  conducted  a  modest  conceptual 
test  at  my  farm.  We  peirked  a  dirty  old  green  pickup  truck  in  the  tall  grass 
not  too  far  from  the  silo  and  we  went  to  a  local  airport  and  leased  a  Cessna 
152.  My  wife  and  I  went  out,  recorded  the  data,  and  correlated  it  with  a 
large  photograph  of  our  farm  area  where  the  terrain  is  much  like  Europe. 

"We  came  up  with  a  set  of  numbers  that  would  indicate  that  you  could 
probably  see  a  tank  from  about  7,000  feet  slant  range.  That  is  less  than 
the  distance  [needed]  to  use  a  Maverick. 

"Many  years  later  .  .  .  those  numbers  were  verified  by  Air  Force  testing. 
The  problem  was  that  the  people  involved  in  the  early  development 
testing  .  .  .  didn't  really  address  the  operational  problem  of  a  pilot  who 
has  to  fire  first  on  a  tank  before  the  air  defense  can  shoot  at  him." 

I  don't  know  what  Mr.  Myers'  conceptual  operational  test  cost. — ^perhaps  a  few 
hundred  dollars.  Yet  had  the  Air  Force  paid  attention  to  his  results,  tney  could  have 
saved  the  hundreds  of  millions  of  dollars  they  wasted  on  a  hopeless  system,  and  in- 
stead gotten  busy  on  an  alternative  that  a  pilot  could  actually  use. 

Section  138  of  Title  10  of  the  U.S.  Code  defines  operational  testing  as: 
"...  the  field  test,  under  realistic  combat  conditions,  of,  any  item  of  {or  key 
component  of)  weapons,  equipment,  or  munitions.  .  .  ." 

As  Special  Counsellor  of  the  House  Armed  Services  Committee  when  that  section 
was  drafted,  I  can  testify  that  the  words  "or  key  component  thereof  were  inserted 
specifically  to  encourage  early  conceptual  operational  testing  rather  than  just  final 
exams  for  completed  weapon  systems. 

Delays  Caused  by  Operational  Testing 

A  third  policy  issue  is  the  charge  that  operational  testing  adds  to  the  already  ex- 
cessive length  of  weapon  development  and  further  delays  fielding  new  weapons. 

As  long  as  the  criterion  is  merely  "fielding  new  weapons",  the  allegation  of  delay 
could  well  be  true.  But  that's  the  wrong  criterion.  The  right  one— just  a  bit  longer — 
is  "fielding  new  weapons  ready  for  combat".  If  operational  testing  shows  that  a 
weapon  isn't  ready  for  combat,  it  should  be  delayed. 

In  fact,  far  from  delaying  the  fielding  of  combat-ready  weapons,  operational  test- 
ing accelerates  the  process  because  it  takes  far  less  time  to  correct  faults  discovered 
early  in  operational  testing  before  production  and  deployment  starts  than  it  takes 
to  retrofit  faulty  weapons  in  the  field.  After  all,  if  you're  just  out  to  deploy  some- 
thing as  fast  as  possible,  you  could  always  cut  out  the  development  testing,  too. 

The  Expense  of  Operational  Testing 

A  fourth  policy  issue  is  the  charge  that  operationsd  testing  costs  too  much,  par- 
ticularly in  an  era  of  declining  budgets.  But  operational  testing  accounts  for  only 
a  tiny  fraction  of  the  total  cost  of  a  weapons  program.  And  every  time  that  it  dis- 
closes a  fault  in  a  new  weapon  that  can  be  corrected  before  (instead  of  after)  the 
production  line  begins  to  roU,  the  saving  swamps  the  relatively  trivial  cost  of  the 
testing. 

Proposal  for  Waivers  of  Operational  Testing 

Mr.  Chairman,  I  cannot  conclude  without  mentioning  the  pending  proposal  to 
allow  waivers  from  current  operational  testing  reauirements.  The  draft  I  have  seen 
would  return  the  balance  between  the  mtikers  ana  the  users  of  weapons  to  the  lop- 
sided inequality  of  a  decade  ago  before  this  committee  led  the  drive  to  level  that 
balance  by  legislating  independent  operational  testing. 

The  sweep  of  this  new  proposal  is  breathtaking.  There  is  no  apparent  limit  to  the 
possible  degradation  or  even  abandormient  of  operational  testing;  the  only  descrip- 
tion of  the  revised  procedures  is  that  they  could  be  "different".  Nor  is  there  any  limit 
to  the  importance  of  the  programs  subject  to  such  procedural  vandalism;  these  new 
provisions  could  apply  to  any  program. 

The  draft  stipulates  four  iustifications  for  such  waivers.  The  first  would  be  if  nor- 
mal operational  testing  is  alleged  to  be  "unreasonably  emensive."  But,  as  I've  noted, 
it  can  cost  a  lot  more  to  fix  a  faulty  weapon  in  the  field  than  it  could  possibly  cost 
to  test  it  properly  before  it  goes  into  production. 

The  second  justification  would  be  if  normal  operational  testing  is  alleged  to  be  im- 
practicable". But  the  lack  of  any  definition  of  the  standards  for  impracticability  is 
just  an  open  invitation  to  hamstring  operational  testing. 
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The  third  justification  would  be  in  cases  where  normal  operational  testing  would 
"cause  unwarranted  delay."  But  the  question  is  delay  in  what?  Delay  in  fielding  a 
weapon,  or  delay  in  fielding  a  combat-ready  weapon? 

The  fourth  justification  is  so  sweeping  that  I  can  only  wonder  why  the  drafters 
even  bothered  with  the  other  three.  This  one  would  allow  a  waiver  if  normal  oper- 
ational testing  would  "be  unnecesseiry  because  of  the  acquisition  strategy  for  that 
system."  As  I  read  that  language,  it  would  let  the  Defense  Department  gut  oper- 
ational testing  any  time  it  wanted  to  by  simply  writing  it  out  of  their  "acquisition 
strategy". 

The  draft  does  conclude  with  one  restriction  on  such  waivers:  they  would  not 
apply  to  programs  "beyond  low-rate  initial  production".  The  clear  implication  is  that 
operational  testing  would  apply  only  to  the  "final  exam",  which  this  language  would 
now  defer  to  the  end  of  low-rate  initial  production.  As  I  noted  earlier,  operational 
testing  should  play  its  role  fi^om  the  very  beginning  of  a  program,  and  not  just  as 
some  "final  exam".  If  the  operational  tester's  hands  are  to  be  tied  until  a  weapon 
completes  low-rate  initial  production,  his  only  choice  will  be  to  run  for  cover  be- 
cause, by  that  time,  the  avalanche  will  be  in  ftdl  force. 

Mr.  Chairman,  this  proposal  is  a  loophole  big  enough  to  drive  a  DFVAD  through. 

Conclusion 

In  conclusion,  Mr.  Chairman,  operational  testing  must  be  independent.  It  should 
be  used  fi"om  the  beginning  of  a  weapon  development.  It  doesn't  delay  deplo5Tnent 
of  weapons  that  work,  it  accelerates  it.  It  doesn't  waste  money,  it  saves  it.  And,  I 
urge  you,  beware  of  proposals  that  would  cripple  the  users'  influence. 
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A  PENTAGON  PRIORITY? 

Operational  Testing  (DOT&E):  An  Office  Without  a  Leader 


Vacancy  Time 
19  months — 

5  months — 


14  months — 
&  counting 


Since  T983, 
the  DOT&E  hos 
been  wifhouf  a 

Director  30% 
of  the  time. 


Reason  tor  Vacancy 

Creation  of  DOT&E  Office  (Sept.  1983) 

until 

confirmation  of  Jack  Krings  (April,  1985) 

Resignation  of  Jack  Krings  (June  1989) 

until 

confirmation  of  Cliff  Duncan  (Nov.  1989) 

Resignation  of  Cliff  Duncan  (Jan.  1993) 
until  ??? 
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